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ABSTRACT 


Numerous  authors  have  proposed  instituting  a  patient 
sorting,  or  triage,  system.  At  the  same  time,  there  is  a 
paucity  of  information  on  the  effect  of  training  and 
experience  on  the  ability  of  persons  to  sort  emergency 
department  patients.  Consequently,  this  study  was  designed 
to  investigate  the  accuracy  of  emergency  department  clerks, 
nurses,  and  physicians  in  classifying  patients'  conditions 
as  emergent,  urgent,  or  nonurgent.  These  classification 
categories  have  been  widely  used  in  the  literature  and  their 
criteria  were  more  comprehensive  than  other  emergency 
department  patient  classifications.  A  second  component  of 
the  study  was  the  examination  of  an  indirect  measure  of  the 
patient's  perception  of  the  urgency  of  his  own  condition. 

The  study  was  carried  out  in  two  Edmonton  emergency 
departments  in  June  1973.  The  nonrandom  patient  sample  was 
restricted  to  patients  seen  by  emergency  physicians  in  no 
more  than  seven  consecutive  24  hour  days.  The  study  was  not 
carried  out  at  the  same  time  in  both  hospitals.  The  clerk's 
assessment  and  the  indirect  measure  of  the  patient's 
perception  of  urgency  was  only  obtained  for  one  of  the  two 
hospitals . 

The  estimate  of  "true  urgency",  which  was  used  to 
calculate  the  accuracy  of  the  staff  and  patient  assessments, 
was  the  rating  assigned  independently  by  at  least  two  of 
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three  physicians  who  reviewed  the  patient  records.  These 
"panel"  physicians  did  not  agree  unanimously  on  their 
urgency  ratings  for  almost  half  of  the  patient  records. 

Most  of  the  patients  were  classified  urgent  or  nonurgent. 

Kendall's  correlation  coefficients,  percentages  of 
agreement,  and  chi  square  goodness  of  fit  tests  were  used 
to  measure  the  agreement  of  the  staff  and  patient  ratings 
with  the  "true  urgency"  estimates.  No  significant 
differences  were  apparent  between  the  accuracy  of  the 
emergency  physician,  nurse,  and  the  indirect  patient  urgency 
ratings.  The  ability  of  the  clerk  closely  approached  that 
of  the  other  staff,  although  she  tended  to  be  more 
conservative  in  her  assessments.  Experience  did  not  appear 
to  influence  the  accuracy  of  the  staff. 

The  sampling  design  of  this  study  was  inadequate 
for  the  generalization  of  the  findings.  While  the  ability 
of  the  staff  to  classify  patients  was  measured,  an  actual 
triage  situation  was  not  simulated. 

The  findings  of  this  study  demonstrate  the  need 
for  refinement  of  "true  urgency"  criteria,  more  extensive 
examination  of  factors  influencing  the  ability  of  staff  to 
classify  patients,  and  further  investigation  into  the 
accuracy  of  the  patients  to  categorize  themselves. 
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CHAPTER  ONE 


INTRODUCTION 

Statement  and  Importance  of  the  Problem 

The  demand  for  service  from  emergency  departments  is 
increasing,  creating  the  need  to  examine  improved  modes  for 
delivery  of  these  services.  One  popular  idea  is  the 
institution  of  a  'triage  officer'  to  sort  patients  and 
direct  them  to  the  most  appropriate  care  center,  whether  it 
be  a  convenience  clinic,  their  own  physician,  the  emergency 
department,  or  some  other  source  of  health  care. 

Up  to  the  present  time,  little  has  been  written  on 
the  training  and  experience  necessary  for  a  triage  officer. 
Therefore,  this  study  was  designed  to  provide  a  foundation 
for  examining  the  effect  of  training  and  experience  on  the 
ability  of  emergency  department  staff  to  classify  patients. 

Summary  of  Methodology 

Clerks,  nurses,  and  emergency  physicians,  hereafter 
referred  to  as  the  "staff",  classified  the  urgency  of 
patients  in  two  emergency  departments.  The  selection  of  the 
patient  sample  was  nonrandomized  and  restricted  to  visits  of 
no  more  than  7  consecutive  24  hour  days  in  June  1973. 
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An  actual  triage  practice  was  not  simulated  for  the 
following  reasons:  (1)  the  staff  had  no  exposure  to  the 

functioning  of  an  actual  triage  practice;  (2)  the  staff 
received  no  instruction  on  classifying  patients,  other  than 
being  provided  with  definitions  and  examples  of  the  urgency 
categories;  and  (3)  the  classification  categories  were 
descriptive  rather  than  practical  triage  referral  options. 
Although  design  steps  could  have  been  taken  to  rectify  the 
latter  two  points,  a  simulated  triage  would  have  limited 
generalizability  because  triage  criteria  are  dependent  upon 
the  health  resources  available  for  referral  and  thus  vary 
somewhat  from  one  hospital  to  another.  Therefore,  urgency 
criteria  (which  have  been  shown  to  be  related  to  triage 
decisions  and  utilized  to  a  greater  extent  than  triage 
criteria  (cf.  Ch.  2))  were  chosen  for  use  in  this  study. 

During  the  course  of  the  study,  each  type  of  staff 
classified  emergency  department  patients  as  'emergent', 
'urgent',  or  'nonurgent'  from  an  initial  observation  of  the 
patients.  The  definitions  of  the  urgency  categories  employed 
were  those  used  by  Vayda,  Gent,  and  Paisley  (1972) .  Along 
with  each  urgency  rating,  the  staff  member  also  indicated 
the  extent  of  his  (or  her)  emergency  department  experience. 

In  addition,  an  indirect  measure  of  the  patient's  perception 
of  the  urgency  of  his  condition  was  obtained  by  having  each 
patient  answer  questions  related  to  the  urgency  criteria. 
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About  two  weeks  later,  a  "panel”  in  each  hospital 
reviewed  up  to  275  patient  records  from  the  respective 
emergency  departments  and  assigned  final  urgency  ratings. 

The  composition  of  the  respective  panels  was  three 
physicians  from  each  of  the  two  hospitals  who  were  familiar 
with  the  functioning  of  the  emergency  department.  Panel 
physicians  I,  II,  and  III  were  from  hospital  A  and  panel 
physicians  IV,  V,  and  VI  were  from  hospital  B.  The 
independent  rating  assigned  by  the  three  physicians  in  each 
hospital  was  used  to  define  the  "true  urgency"  of  the 
patient.  This  "true  urgency"  rating  was  then  used  as  a 
criterion  for  purposes  of  estimating  the  ability  of  the 
staff  to  classify  patients.  Finally,  information  (which  the 
investigator  believed  might  influence  the  accuracy  of  the 
staff  assessments)  was  also  abstracted  from  the  patient 
records  when  they  were  retrieved  for  the  panel  review. 

Limitations  of  the  Study 


Internal  Validity 

Factors  were  present  in  this  study  which  may  have 
jeopardized  the  interpretation  of  the  findings. 

Firstly,  while  all  staff  members  participating  in  the 
study  were  asked  to  become  familiar  with  and  employ  the 
predefined  urgency  criteria,  there  is  no  assurance  that 
individual  staff  members  did  not  disregard  these  definitions 
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and  use  their  own  interpretation  of  the  terms  'emergent', 
'urgent',  and  'nonurgent'. 

Secondly,  the  staff  members  were  to  assign  urgency 
ratings  to  the  patients  without  the  knowledge  of  another 
staff  member's  rating.  While  steps  were  taken  to  maintain 
this  independence,  the  investigator  is  not  assured  that 
inter-staff  collaboration  was  entirely  eliminated.  However, 
since  the  staff  were  exceptionally  busy  during  the  time  of 
the  study,  consultations  were  probably  minimal.  On  the  other 
hand,  this  increased  workload  may  have  created  the 
undesirable  effect  of  curtailing  the  staff's  actual  ability 
to  classify  patients. 

Thirdly,  "panel"  members  (cf.  p.  3)  were  limited  in 
their  final  assignment  of  an  urgency  rating  by  being 
restricted  to  the  patient  records.  Lab  and  x-ray  results 
were  missing  from  some  patient  charts,  and  no  evaluation  of 
the  patient's  condition  after  he  left  the  emergency 
department  was  available. 

External  Validity 

Definitive  conclusions  as  to  the  effect  of  experience 
and  training  on  the  ability  of  clerks,  nurses,  and 
physicians  would  have  necessitated  the  selection  of  a  random 
sample  of  registration  clerks,  nurses,  and  physicians  from 
the  population  to  which  one  wished  to  generalize  (e.g.. 
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clerks,  nurses,  and  physicians  in  Edmonton  emergency 
departments) .  Since  this  was  not  possible,  the  investigator 
chose  to  carry  out  the  study  using  nonrandom  samples,  but 
replicating  the  study  to  see  if  a  common  pattern  emerged.  In 
other  words,  this  design  does  not  allow  one  to  assume  that 
the  staff's  ability  in  any  one  hospital  has  any  meaning  for 
the  staff  in  another  hospital;  but  by  carrying  out  the  same 
study  in  more  than  one  hospital,  a  measure  of  inter-hospital 
variation  in  staff  perception  was  obtained.  Here  too,  it 
must  be  remembered  that  the  hospitals  studied  were  not 
randomly  selected. 

The  investigator  was  limited  to  assessing  the 
capabilities  of  the  three  types  of  emergency  staff  (clerk, 
nurse,  emergency  physician)  in  classifying  patients 
according  to  the  urgency  of  their  conditions.  These  staff 
members  had  no  formal  initiation  into  this  role.  Therefore, 
the  investigator  wishes  to  point  out  that  with  training, 
experience  and  practical  triage  alternatives,  the  level  of 
judgement  of  the  various  types  of  staff  might  be 
significantly  altered.  Baldridge  (1966)  claims  that  a  triage 
nurse  improves  her  skill  by  knowledge  gained  from  her 
mistakes . 

The  utility  of  the  data  which  was  collected  from  the 
patient  records  is  limited  to  accounting  for  variations 
observed  in  assigning  urgency  ratings  since:  (1)  The  sample 
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studied  was  consecutive  patients  seen  by  emergency 
physicians  in  a  very  short,  summer  time  period.  Therefore, 
the  sample  was  not  representative  of  all  services  being 
provided  by  the  emergency  department  for  an  entire  year.  (2) 
The  study  was  carried  out  while  the  emergency  department  of 
a  third  hospital  was  closed.  Thus,  the  caseloads  of  the  two 
hospitals  studied  were  probably  atypical. 

Definition  of  Terms 


An  emergency  department  is 

the  facilities  and  services  provided  primarily 
for  the  management  of  outpatients  coming  to 
the  hospital  for  treatment  of  conditions 
determined  clinically  or  considered  by  the 
patient  or  his  representative  to  require 
immediate  medical  care  in  the  hospital 
environment  (American  Hospital  Association, 
1972,  p.  viii) . 

An  emergent  condition 

requries  immediate  medical  attention;  time 
delay  is  harmful  to  patient;  disorder  is  acute 
and  potentially  threatening  to  life  or 
function. 

Cases  of  this  type  will  usually  require 
hospitalization  (Vayda,  Gent,  &  Paisley,  1972, 
P-  6)  . 

An  urgent  condition 

requires  medical  attention  within  a  period  of 
a  few  hours;  there  is  possible  danger  to 
patient  or  to  ultimate  outcome  if  not  promptly 
medically  attended;  disorder  is  of  acute  onset 
but  not  necessarily  severe  or  life 
threatening. 

Will  usually  not,  but  may  require 
hospitalization.  (Vayda,  Gent,  &  Paisley, 
1972,  p.  6) . 
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A  nonurgent  condition 


does  not  require  the  resources  of  an  emergency 
service;  symptoms  of  long  duration  without 
sudden  change  in  severity;  referral  for 
routine  medical  care  is  all  that  is  needed; 
disorder  is  not  acute  and  minor  severity. 

Routine  care  in  a  physicians  office  or  no 
medical  care  required  (Vayda,  Gent,  &  Paisley, 

1972,  p.  6) . 

A  registration  clerk  is  a  person  with  no  medical  or  nursing 
training  who  is  employed  to  record  demographic  patient 
information  for  an  emergency  department. 

A  nurse  is  a  registered  nurse  as  defined  by  The  Registered 
Nurses  Act  (R.S.A.  1955,  c.  283)  who  renders  nursing  care 
full  or  part  time  in  an  emergency  department. 

An  emergency  physician  is  a  medical  practitioner  as  defined 
by  The  Medical  Profession  Act  (R.S.A.  1955,  c.  198)  who  is 
responsible  for  full  or  part  time  coverage  of  an  emergency 
department. 

Triage  means  an 

immediate,  brief  medical  evaluation  of  all 
incoming  patients,  determination  of  the 
general  nature  of  the  problem  and  the  kind  of 
service  needed  and  appropriate  referral 
(Weinerman  &  Edwards,  1964,  p.  57). 

A  triage  officer  is  one  who 

interviews  each  patient  who  arrives  in  the 
emergency  department,  assesses  his  presenting 
problem,  and  determines  priorities  in  securing 
care  expeditiously  (O'Boyle,  1972,  p.  1394) . 


Summary 


By  examining  the  relationship  between  (1)  the  ability 


of  the  staff  to  assess  the  urgency  of  patients  in  emergency 
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departments,  and  (2)  the  type  of  training  and  length  of 
experience  of  these  staff  members,  it  was  hoped  that  this 
study  would  provide  information  that  would  assist  in 
developing  training  and  experience  standards  required  for 
triage  officers.  Before  discussing  this  study  in  further 
detail,  literature  pertinent  to  this  investigation  will  be 
presented. 
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CHAPTER  TWO 


SOME  RELEVANT  LITERATURE 
Need  for  a  Classification  System 

For  many  years  concern  has  been  expressed  over  the 
increasing  demands  being  made  of  emergency  departments 
(Webb,  1969,  pp.  13-15).  Since  this  phenomenon  is  not 
expected  to  disappear  (Roth,  1971,  p.  319;  Gibson,  1973a), 
many  solutions  have  been  proposed  for  relieving  this 
expansion  problem;  most  of  which  require  that  'nonurgent' 
patients  receive  differential  treatment.  Other  reasons 
advanced  for  decreasing  the  use  of  emergency  departments  for 
the  treatment  of  nonurgent  conditions  are  that  this  would 
remove  one  source  of  staff  dissatisfaction  (Bainbridge, 

1972) ,  and  insure  a  more  appropriate  use  of  emergency 
resources  (Fahey,  1964,  p.  96;  Westcott,  1971). 

As  early  as  the  1950 's,  it  was  realized  that 
emergency  rooms  were  providing  medical  care  to  patients  with 
nonurgent  conditions  and  not  being  used  solely  for  acute 
conditions  of  recent  onset  (Howell  &  Buerki,  1957;  Reifler, 
1957;  Shortliffe,  Hamilton,  &  Noroian,  1958).  During  the 
1950 's,  these  authors  proposed  that  emergency  rooms  should 
be  expanded  to  serve  all  persons  seeking  care. 
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Since  then,  however,  attempts  have  been  made  to 
control  the  expansion  of  emergency  facilities  by  educating 
the  public  not  to  misuse  emergency  services  (Skudder, 
McCarrol,  &  Wade,  1961;  Fahey,  1964,  pp.  98-103;  Silver, 
Manegold,  &  Gartland,  1966;  Bain  &  Johnson,  1971; 

Bainbridge,  1972),  although  in  one  study  this  practice  " . . . . 
alienated  some  patients  and  opened  the  hospital  to  criticism 
[Kirkpatrick  &  Taubenhaus,  1967,  p.  22]."  Baltzan  (1972) 
reports  that  the  introduction  of  utilization  fees  appears  to 
have  no  effect  in  limiting  the  demands  made  of  the  emergency 
department. 

Many  authors  (Lee,  Solon,  &  Sheps,  1960;  Kluge, 
Wegryn,  &  Lemley,  1965;  Beloff,  1968;  Booker  &  Vansant, 

1969;  Rosenfeld,  1971,  pp.  IX  63-69  &  VIII  39;  Roy,  1971; 
Hart,  1972;  Gibson,  1973b)  have  favored  the  development  of 
comprehensive  ambulatory  care  programs  to  meet  the  wide 
variety  of  service  demands  being  made  on  emergency 
departments.  In  an  attempt  to  provide  care  for  nonemergent 
conditions,  for  which  emergency  departments  have  been 
deemed  inadequate  (Kirkpatrick  &  Taubenhaus,  1967;  Brook  & 
Stevenson,  1970;  Jonas,  1971;  Nahum,  1971) ,  facilities  both 
within  the  hospital  (Jacobs,  Gavett,  Raquso,  &  Thurber, 

1972;  Morgan,  1972;  Gibson,  1973a;  Taubenhaus,  1973)  and  in 
the  community  (Paris  &  Callan,  1968;  Brown,  1971; 

Hochheiser,  Woodword  &  Charney,  1971;  Moore,  Bernstein,  & 
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Bonanno,  1972)  have  been  proposed. 

Whatever  the  sources  of  this  convenience  service  may 
be,  a  means  of  sorting  patients  for  such  care  must  be 
formulated.  While  various  authors  cite  the  need  for  an 
effective  triage  program  (Weinerman  &  Edwards,  1964; 
Estampador,  Kessler  &  Scott,  1968;  Schwartzman,  1971,  p.  22; 
Stewart,  Robertson,  Chipman,  &  Gordon,  1971,  p.  51;  Gibson, 
1973a;  Lowe  &  Baker,  1973;  Taubenhaus  &  Simon,  1973),  few 
studies  have  suggested  sorting  criteria  and  the  level  of 
training  and  experience  necessary  to  use  this  criteria. 

Development  of  Screening  Criteria 

Attempts  to  classify  emergency  department  patients 
are  prevalent  in  the  literature  on  emergency  services. 

Vaughan  and  Gamester  (1966)  devised  an  illness 
classification  system  based  on  a  trauma-nontrauma  dichotomy. 
Reed  and  Reader  (1967)  classified  patients  according  to  the 
severity  of  their  conditions.  Alpert,  Kosa,  Haggerty, 
Robertson,  and  Heagarty  (1969)  investigated  groups  of 
emergency  department  patients  on  the  basis  of  the  stability 
of  their  hospital  and  doctor  relationships.  Booker  and 
Vansant  (1969)  labelled  visits  as  necessary  or  unnecessary. 
Garraway  (1969)  assessed  the  illnesses  and  injuries  presented 
by  patients  as  very  serious,  serious,  quite  serious,  moderate, 
or  minor.  Teglas  (1969)  defined  patients  as  minor, 
intermediate,  or  major  emergencies;  major  emergencies  were 
those  which  presented  a  threat  to  the  patient's  life  or  well 
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being,  minor  emergencies  presented  no  such  threat,  and  the 
intermediate  group  was  comprised  of  borderline  cases.  Hess 
(1970)  used  the  terms  emergency  and  quasi  emergency.  Roth, 
Douglas,  and  Howe  (1971)  described  patients  according  to  the 
emergency  department  staff's  perception  of  the  legitimacy  of 
the  demand  for  service.  Recently,  Allen,  Barnard,  Falk, 
Higgs,  and  McCracken  (1973)  have  employed  five  categories  of 
severity:  extreme  emergencies,  acute  emergencies,  minor 
emergencies,  trivia,  and  elective. 

The  most  common  classification  is  an  emergent,  urgent 
or  nonurgent  assessment  (Lee,  Solon,  &  Sheps,  1960;  Lavenhar 
Ratner,  &  Weinerman,  1968;  Robinson,  Kinnis,  Anderson,  Argue 
&  Miller,  1969;  Webb,  1969;  Perkoff  &  Anderson,  1970;  White 
&  O'Connor,  1970;  Bain  &  Johnson,  1971;  Vayda,  Gent,  & 
Paisley,  1972;  Chipman,  Reid,  &  Gordon,  1973;  Davis,  1973; 
McDonald,  McDonald,  Steinmetz,  Enterline,  &  Salter,  1973; 
Walker,  1973),  or  variations  of  these  categories:  emergency, 
nonemergency  (Read,  1966) ;  emergency,  nonemergency,  did  not 
need  to  be  seen  (Kirkpatrick  &  Taubenhaus,  1967);  emergent, 
urgent,  elective  (Trier  &  Levy,  1969) ;  and  urgent,  semi- 
urgent,  non-urgent  (Fox  &  Mullany,  197  3)  .  It  must  be  noted 
that  such  classifications  were  designed  to  describe  the 
emergency  department  patient  mix  and  not  as  screening 
criteria  the  latter  being  a  function  of  the  referral  options 
available  in  each  emergency  department  (O'Boyle,  1972). 


■-  '  :  ■  ■  :  '  1  '  O  •  X  1  1 


.  m  >  -•  u  •  it  vr>n<  ■••.'T9  am  ii  -ft*  '  ■■  (  **  ' 


»o.‘;i3r-  .-wToms  t aaionsp-some  9iy??x*xe?  :  rxavss 

. 


4  ,  in  90  v/?bv..V  i  [V?I  (ncTf.  >T  *J  a  :t-V  ,  V;.  t  - 

' 


t  turtle  8  v9niI*$dTiS  %*s  x*nn  r.sjg  ,  hi  s  .oC  oM  *bl^ao 


bo*on  rd  lsx.  i  Jic 


•  ■  V  >C.  •■  '•■■  -  •  •  ••  -Li  .Tv.  -.  '•  .  {!\.: 


A  few  studies  have  presented  triage  criteria  for 
screening  patients  arriving  at  an  emergency  department. 

Walker  (1973)  listed  symptomatic  critesria  for  use  in 
limiting  emergency  care  to  those  conditions  of  "  ... 
sufficient  magnitude  to  necessitate  the  use  of  an  emergency 
room  [p.  28]."  The  categorization  of  patients  made  by  using 
these  criteria  was  related  to  an  emergent-urgent,  nonurgent 
dichotomy  at  a  level  of  significance  of  0.05  (p.  29).  Vayda, 
Gent  &  Paisley  (1973a,  1973b)  proposed  criteria  for  triage 
decisions  which  were  based  on  a  ranking  of  seriousness  of 
presenting  complaints.  Here  too,  a  relationship  was  observed 
between  decisions  based  on  the  triage  criteria  and  an 
emergent-urgent,  nonurgent  classification. 

Boyd,  Mains,  &  Flashner  (1972)  have  described  a 
Critical  Injury  Index.  Kirkpatrick  and  Youmans  (1971; 
Kirkpatrick,  1972)  have  designed  and  tested  a  Trauma  Index. 
These  indexes  were  designed  so  that  ambulance  attendants, 
police  officers  and  the  public  could  determine  the  most 
appropriate  level  of  facility  required  by  an  accident 
victim. 

Who  is  capable  of  Classifying  Emergency  Department  Patients? 

While  outlining  the  statewide  Illinois  Trauma 

Program,  Boyd,  Mains,  and  Flashner  (1973)  emphasized  that 

a  major  commitment  must  be  made  to  the  early 
separation  of  all  patients  with  critical 
injuries  or  illnesses  from  the  larger  group  of 
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outpatients  who  have  beleagured  today's 

emergency  departments  [p.  282] . 

In  Illinois,  law  enforcement  officers  and  ambulance 
attendants  initiate  transport  to  the  most  appropriate 
treatment  center  for  accident  victims.  These  authors  decry 
the  practice  of  using  anyone  but  the  most  experienced  and 
capable  individual  as  a  triage  officer  in  the  emergency 
department,  but  offer  no  proof  that  the  practice  of  using 
nonprofessionals  is  inadequate. 

Some  emergency  departments  use  a  physician  as  a 
triage  officer  (Weinerman  &  Edwards,  1964;  Saxl,  1970) . 

While  describing  a  triage  case  study,  Beloff  (1968)  states 
that  "  ...  the  most  important  requisite  is  a  comprehensive, 
competent,  and  interested  physician  to  do  triage  [p.  69]." 
Others  feel  less  qualified  persons  may  be  as  effective 
(Baldridge,  1966;  Canizaro,  1971).  Taubenhaus  and  Simon 
(1973)  report  that  in  a  small  emergency  department  in  a 
resort  area  "  ...  triage  is  performed  by  every  member  of  the 
department  from  clerk  to  physician  [p.  43]." 

Jones  &  Shires  (1969)  state  that  nurses  make  correct 
triage  decisions  98%  of  the  time.  Canizaro  (1971)  reports 
that  nurses  send  less  than  5%  of  patients  to  the  wrong  area. 
However,  qualifications  for  a  triage  nurse  are  not  well 
established.  Slater  (1970,  p.  52)  advises  a  minimum  of  3 
months  orientation  before  a  nurse  is  ready  to  accept  the 
responsibilities  of  a  triage  nurse,  while  Jones  and  Shires 
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(1969)  and  Canizaro  (1971)  state  that  nurses  with  several 
yeaifs  experience  are  required.  Folse  (1971)  and  Rudolf 
(19'71)  maintain  that  a  well-trained  nurse  can  adequately 
carry  out  this  role.  Hart  (1972)  and  Vayda,  Gent,  and 
Paisley  (1973a,  1973b)  see  triage  as  a  possible  role  for 
nurse  practitioners. 

Recently  there  has  been  interest  in  involving 
patients  in  a  more  active,  informed,  participating  role  in 
their  health  care  (Nader  group  plans  manual  for  hospital 
users,  1973;  Schlesinger,  1973)  and  in  investigating  the 
opinions  of  the  consumers  of  ambulatory  health  services 
(Sussman,  Caplan,  Haug,  &  Stern,  1967;  Jesion,  1970;  Davis, 
1971) .  It  seems  expedient,  therefore,  to  also  assess  how  the 
patient  views  the  urgency  of  his  condition.  The  recent  study 
of  Maddox  and  Douglass  (1973)  noted  that  patient  self-ratings 
were  better  predictors  of  subsequent  physician  ratings  than 
vice  versa.  If  indeed  patients  are  better  able  to  predict 
their  future  health  needs,  this  lends  further  support  to 
soliciting  patients’  opinions  on  the  urgency  of  their 
condition. 

Some  statements  on  utilizing  and  examining  patient 
judgements  are  available.  O'Connor  (1972)  has  proposed  a 
system  of  self-triage.  Downey  (1973)  reports  that  a  large 
sign  in  the  emergency  room  explaining  the  function  of  an 
adjacent  walk-in  ambulatory  clinic  enables  most  patients  to 
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triage  themselves.  Bain  and  Johnson  (1971)  used  similar 
urgency  scales  for  a  physician-patient  comparison  of  urgency 
assessments.  It  is  interesting  to  note  that  the  patients 
viewed  more  cases  as  nonurgent  than  did  the  physicians. 
Garraway  (1969)  reported  that  61.5%  of  patients  gave  a 
similar  assessment  to  the  Casualty  Officer  regarding  the 
seriousness  of  their  injury.  Davis  (1971,  pp.  87-93) ,  while 
examining  variations  in  perception  of  urgency  by  patients 
with  nonurgent  conditions,  developed  a  composite  score  of 
perceived  urgency  comprised  of  feeling  state,  interference 
with  function,  and  extent  of  worry  components.  She  stated 
that : 

At  best,  the  composite  score  of  perceived 
urgency  is  a  crude  index  of  symptom 
perception,  and  neither  the  validity  or  the 
reliability  of  the  measurement  tool  were 
ever  established.  [Davis,  1971,  p.  103] . 

Although  it  is  believed  that  patients  and  health 
providers  do  not  have  the  same  perception  of  urgency  (Hess, 
1970;  Coleman  &  Errera,  1963),  few  attempts  have  been  made 
to  determine  if  even  professionals  view  urgency  in  the  same 
manner.  While  no  measure  of  validity  was  reported  for  most 
studies  where  a  physician  or  nurse  assigned  an  urgency 
rating  to  patients  based  on  a  record  review  (Lee,  Solon,  & 
Sheps,  1960;  Gala,  1969;  Webb,  1969;  Perkoff  &  Anderson, 
1970;  Jacobs,  Gavett,  &  Wersinger,  1971;  Kirpatrick  & 
Taubenhaus,  1967),  two  studies  did  include  an  independent 
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urgency  reassessment.  Vayda  reviewed  an  11%  sample  of 
patient  charts  which  had  been  previously  categorized  as 
emergent,  urgent,  or  nonurgent  by  emergency  physicians.  He 
observed  an  80%  agreement  between  himself  and  the  emergency 
physicians  (Vayda,  Gent,  &  Paisley,  1972,  p.  9).  Ambuel, 
Cebulla,  Watt,  and  Crowne  (1964) ,  using  urgency  definitions 
based  on  the  need  for  immediate  follow-up  of  the  patient, 
reported  inter- judge  reliability  coefficients  of  0.74  and 
0.92  in  an  independent  physician  review  of  two  samples  of 
146  and  139  patient  records,  respectively.  Davis  (1971,  pp. 
40,  41)  found  these  urgency  criteria  to  be  lcLcking  face 
validity  when  she  used  them.  Therefore,  she  did  not  analyse 
data  gathered  with  this  instrument. 

Summary 

To  effectively  utilize  emergency  departments,  a 
number  of  authors  have  suggested  the  institution  of  triage. 
Up  to  the  present  time,  evaluation  of  screening  criteria  and 
the  qualifications  of  triage  officers  has  been  very  limited, 
even  though  there  appears  to  be  much  variation  in  the 
literature  on  who  is  capable  of  carrying  out  this  triage 


function . 
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CHAPTER  THREE 


METHODOLOGY 

The  first  part  of  this  chapter  describes  an  optimal 
design  for  investigating  the  effect  of  training  and 
experience  on  the  ability  of  emergency  department  staff  to 
classify  the  urgency  of  patients.  Since  compromises  in  the 
optimal  design  were  necessary  for  implementation  of  this 
study,  the  remainder  of  this  chapter  is  devoted  to  noting 
the  changes  that  were  made  for  the  actual  method  employed. 

Figure  1  is  provided  to  assist  the  reader  in 
understanding  the  following  presentation  (cf.  pp.  18-29)  of 
an  optimal  design  which  is  considered  to  be  the  minimal 
requirements  for  eliciting  valid  results.  The  design  that 
was  implemented  (cf.  pp.  29-37)  did  not  meet  all  the 
requirements  of  this  optimal  design. 

Optimal  design 

A  review  of  the  literature  (cf.  ch.  2)  shows  that  an 
emergent,  urgent,  nonurgent  classification  is  the  most 
widely  accepted  classification;  and  since  Vayda,  Gent,  and 
Paisley  (1972)  have  presented  the  most  comprehensive 
description  of  these  terms,  their  classification  is 
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Specifications 


Instrument 


a 


Randomly  select  emergency 
departments  (E.D.)  from 
population  of  interest. 


V 


b 


Doctors  from  E.D.  review  and 
recommend  improvements  in  the 
criteria  for  emergent,  urgent, 
nonurgent  conditions. 


See  Appendix  A  for 
classification  of 
emergent,  urgent  and 
nonurgent  conditions 


V 


c 


Pretest  method  in  hospitals; 
modify  design  and  repeat 
pretesting,  if  necessary. 


V 


d 


Obtain  a  random  sample  of  a 
minimum  of  250  patients  (pt.) 
stratified  by  hospital, 
time  of  day,  day  of  week, 
month  of  year. 


V 


e 


Clerk  registering  patient 
-assesses  urgency  of  patient 
-notes  how  long 
she  has  worked  in  an  E.D. 
-marks  pt.  chart  number  on  all 
rating  forms 

-attaches  nurse  &  physician 
rating  forms  to  pt.  chart  . 


See  Appendix  B-l  for 
clerk's  rating  form 


V 


Figure  1:  Optimal  design 
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See  Appendix  B-2  for 
nurse's  rating  and 
patient  question  form 


Nurse  accompanying  patient  to 
examination  area 
-asks  pt.  questions 
-assesses  urgency  of  patient 
-notes  how  long  she  has 
worked  in  an  E.D. 

-detaches  form  from  pt.  chart* 


V 


g 


Emergency  physician  who 
examines  patient 
-assesses  urgency  of  patient 
-notes  how  long  he  (she)  has 
worked  in  an  E.D. 

-detaches  form  from  pt.  chart. 


See  Appendix  B-4  for 
emergency  physician 
rating  form 


V 


h 


After  selected  patients  are 
rated  by  staff,  wait  3-10  days  . 


V 


1 


Pt.  charts  are  retrieved  for 
all  patients  in  study; 
urgency  rating  assigned  to 
cases  by  panel  of  3  doctors  . 


See  Appendix  B-5  for 
panel  rating  form 


V 


j 


Data  coded  from  pt.  records  . 


See  Appendix  B-6 


Figure  Is  Optimal  design  (continued) 
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recommended  as  the  basis  for  the  patient  urgency  ratings. 
However,  to  ensure  that  these  urgency  definitions  and 
examples  are  valid  for  the  emergency  departments  which  have 
been  randomly  selected  from  all  emergency  departments  of 
interest  to  the  investigator  (Figure  l.a),  it  is  essential 
that  the  classification  be  reviewed  and  revised  by  persons 
knowledgeable  of  the  type  of  service  being  given  in  each 
emergency  department  selected  (Figure  l.b) . 

To  acquaint  the  staff  with  the  study  methods  and  to 
identify  potential  sources  of  invalidity,  a  pretest  of  the 
data  collection  procedure  is  advisable.  If  modifications  in 
design  are  necessary,  they  should  be  made  and  the  pretest 
repeated  (Figure  l.c) . 

The  sample  should  include  a  minimum  of  250  patients, 
randomized,  clustered,  and  stratified  (Figure  l.d).  At  least 
250  subjects  are  needed  to  attain  power  of  0.90  for  the  chi 
square  goodness  of  fit  test,  assuming  a  small  effect  size, 
and  a  level  of  significance  of  0.05,  with  2  degrees  of 
freedom  (Cohen,  1972,  p.  228).  Figure  2  outlines  an 
appropriate  sampling  design  for  stratifying  by  variables 
which  have  been  found  to  be  related  to  the  urgency  of 
patients,  specifically,  the  type  of  hospital  (Davis,  1971, 
p.  13;  Davis,  1973,  p.  85),  time  of  day  (Gala,  1969,  p.  32; 
Bain  &  Johnson,  1971,  p.  36) ,  and  day  of  the  week  (Gala, 
1969,  p.  29;  Bain  &  Johnson,  1971,  p.  36).  Although,  no 
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studies  have  been  found  which  examined  changes  in  patient 
urgency  by  the  time  of  the  year,  some  studies  (Barth,  1967; 
Roy,  1971,  p.  42;  Chipman,  Reid,  &  Gordon,  1973),  reported 
variations  in  volume  by  season.  If  the  same  number  of 
patients  are  selected  from  time  periods  of  unequal  patient 
load,  the  responses  from  the  samples  must  be  weighted  in  a 
manner  which  ensures  that  no  time  period  is  over  or  under 
represented.  For  example,  assume  that  it  is  known  that  on 
the  average  twice  as  many  patients  are  seen  during  five 
weekdays  as  are  seen  during  the  weekend.  Accordingly,  if  ten 
patients  were  selected  from  weekday  strata  and  ten  from 
weekends  for  one  season  in  one  hospital,  then  the  weekday 
responses  should  be  multiplied  by  a  factor  of  two.  Analogous 
weightings  are  also  required  for  those  seasonal  and  peak 
strata  designated  in  Figure  2.  Systematic  random  sampling 
within  any  one  cell  of  Figure  2  is  recommended  when  more 
patients  are  expected  during  a  selected  time  period  than 
the  investigator  wishes  to  sample.  For  example,  if  one 
wishes  to  study  five  patients  but  fifteen  are  likely  to 
appear ,  then  every  third  patient  should  be  selected, 
after  randomly  choosing  which  of  the  first  three  patients 
entering  the  emergency  department  will  be  the  first  study 
patient . 
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Figure  2:  Optimal  sampling  design 
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The  clerk  registering  a  selected  patient  records  her 
opinion  of  the  condition  of  the  patient  —  i.e., 
emergent,  urgent,  or  nonurgent  (Figure  l.e)  —  on  the 
assessment  form  provided  (Appendix  B-l) .  On  this  same  form 
she  also  notes  how  long  she  has  worked  in  an  emergency 
department  (e.g.,  less  than  3  months,  3  months  to  1  year,  1 
to  2  years,  or  more  than  2  years) .  The  unit  of  measurement 
for  experience  must  be  kept  the  same  for  the  nurse  and 
emergency  physician  as  well,  so  that  comparisons  can  be  made 
between  types  of  staff  on  the  effect  of  experience.  If 
grouped  data  is  used,  the  first  category  is  defined  in  terms 
of  the  length  of  the  probation  period  for  new  staff  and  is 
therefore  an  appropriate  novice  category.  The  last  category 
for  experience  is  defined  in  terms  of  the  minimal  time 
required  to  become  very  experienced  in  handling  emergency 
patients.  (The  consensus  of  staff  members  in  this  study  was 
that  people  who  had  worked  in  an  emergency  department  for  at 
least  two  years  were  very  familiar  with  the  operation  of  the 
department.)  Other  experience  categories  (e.g.,  3  months  to 
1  year,  1  to  2  years)  are  arbitrarily  chosen  in  an  attempt 
to  approximate  cells  of  equal  and  sufficient  size  to  permit 
significance  testing  of  the  data  collected.  The  reader 
should  note  that  the  use  of  categories  to  designate  length 
of  experience  (1)  restricts  the  amount  of  information  one  is 
able  to  obtain  on  the  effect  of  experience,  but  (2) 
facilitates  uniformity  and  ease  for  respondents. 
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The  clerk,  after  registering  the  patient,  places  the 
patient  chart  number  on  each  data  collection  form  to  enable 
collation  at  a  later  date.  She  then  attaches  the  nurse  and 
physician  assessment  forms  (Appendix  B-2  &  B-4)  to  the 
patient  record  and  places  her  completed  assessment  form  in  a 
box  provided  at  her  desk. 

It  is  unreasonable  to  expect  patients  to  familiarize 
themselves  with  the  complete  criteria  for  emergent,  urgent, 
and  nonurgent,  and  then  categorize  themselves.  Furthermore, 
such  an  approach  is  fraught  with  problems  of  attaining 
internal  validity,  particularly  for  those  patients  with 
nonurgent  conditions.  As  a  consequence,  it  is  recommended 
that  the  nurse  pose  standardized  questions  (Figure  l.f),  the 
answers  to  which  will  allow  the  researcher  to  categorize  the 
patient  into  urgency  categories.  It  is  assumed,  however, 
that  these  placements  are  made  by  the  patients. 

By  reviewing  the  urgency  classification  (Appendix  A) , 

the  following  criteria  are  recommended  as  a  suitable  basis 

for  formulating  the  patient  questions: 

emergent:  . . .  requires  immediate  medical  attention 

...  will  usually  require  hospitalization 
urgent:  ...  requires  medical  attention  within  a  few  hours 

. . .  will  not  usually  . . .  require  hospitalizaiton 
nonurgent:  ...  symptoms  of  long  duration 

...  routine  medical  care  is  all  that  is  needed 

For  example,  the  following  standardized  questions  are  (at  a 

face  validity  level)  a  reflection  of  the  above  criteria  and 
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may  provide  an  indirect  measure  of  the  patient's  perception 
of  the  urgency  of  his  condition. 

1.  How  long  has  the  patient  been  aware  of  the  problem  which 
brought  him/her  to  the  emergency  department  today? 

—  less  than  6  hours  —  1  to  7  days 

—  6  to  24  hours  —  more  than  a  week 

2.  a  Does  the  patient  think  that  he/she  may  have  to  stay  in 
hospital? 

—  no  —  yes 

IF  NO,  ASK: 

2.b  If  the  patient  could  have  obtained  a  doctor's 
appointment  within  the  next  3  days,  would  this  have  been  as 
convenient  for  him/her  as  coming  here  today? 

--  no  —  yes 

The  responses  to  these  questions  could  be  recoded  by 
criteria  such  as:  (1)  patients  consider  themselves  to  be 
"emergent"  if  they  have  been  aware  of  their  problem  for  less 
than  24  hours  and  answer  "yes"  to  question  2. a;  (2)  patients 
consider  themselves  "urgent"  if  they  have  been  aware  of 
their  problem  for  more  than  24  hours,  and  answer  "yes"  to 
question  2. a,  or  have  had  their  problem  for  less  than  24 
hours  and  answer  "no"  to  question  2.b;  and  (3)  patients 
consider  themselves  "nonurgent"  if  they  have  had  their 
problem  for  more  than  24  hours,  and  answer  "no"  to  question 
2.b  or  have  answered  "yes"  to  question  2.b. 

After  recording  the  patient  responses,  the  nurse 
completes  her  assessment  of  the  urgency  of  the  patient's 
condition  and  indicates  how  long  she  has  worked  in  an 
emergency  department.  She  then  detaches  her  assessment  form 
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from  the  patient  chart  and  places  it  in  a  box  provided  at 
the  nursing  station. 

Next,  the  emergency  physician  examining  the  patient 
assigns  an  urgency  rating  to  the  patient  before  x-ray  or 
laboratory  results,  or  consultation  has  been  obtained.  The 
physician  also  indicates  the  length  of  time  he  (she)  has 
worked  in  an  emergency  department  (Figure  l.g) .  The 
emergency  physician  then  removes  this  assessment  form  from 
the  patient  chart  and  places  it  in  a  box  provided  at  the 
doctor's  desk. 

No  staff  member  sees  the  urgency  rating  assigned  by 
another  staff  member,  in  order  that  all  staff  assessments 
are  kept  independent.  If  this  was  not  the  case,  staff 
members  would  have  more  information  (i.e.,  ratings  of  other 
staff)  than  they  would  have  in  an  actual  triage  situation. 

After  enough  time  has  elapsed  for  the  laboratory  and 
x-ray  results  to  be  filed  with  the  patient  charts  (in  this 
study  a  minimum  of  3  days  was  considered  necessary  by  . 
medical  records  personnel) ,  (Figure  l.h) ,  the  patient  charts 
are  retrieved  for  review  by  a  minimum  of  three  physicians  in 
each  hospital  (Figure  l.i).  These  physicians  should  be 
familiar  with  the  functioning  of  an  emergency  department, 
and  must  have  experience  in  treating  ambulatory  patients. 


Since  the  primary  objective  of  this  study  is  to 
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determine  the  accuracy  of  staff  in  assessing  urgency  of 
patients'  conditions,  the  "true  urgency"  of  each  patient  is 
required.  These  "true  urgency"  assessments  should  only  be 
made  after  the  results  of  laboratory  tests  and  x-rays  are 
known  and  the  patient  diagnosed.  Consequently,  a  review  of 
patients'  records  by  physicians  appears  to  be  a  reasonable 
approach  to  attaining  these  assessments.  If  only  one 
physician  assigns  urgency  ratings,  no  measure  of  the 
reliability  or  validity  of  this  physician's  estimate  is 
available.  Two  physicians  are  also  inadequate  since  cases 
(where  differences  are  encountered  between  the  two 
physicians)  would  have  to  be  excluded  from  the  study. 
Accordingly,  the  results  of  such  a  study  could  not  be 
interpreted  because  the  sample  would  not  include  patients 
who  were  difficult  for  the  "panel"  (cf.  p.  3)  to  rate.  By 
including  at  least  a  third  physician,  a  final  urgency  rating 
can  be  obtained  for  more  patients  by  choosing  the  rating 
assigned  by  at  least  two  physicians.  By  measuring  the  inter¬ 
judge  consistency  in  assigned  ratings,  one  is  able  to  obtain 
a  measure  of  reliability.  Although  the  actual  validity  of 
the  assumption  of  "true  urgency"  is  not  available,  it  is 
theoretically  impossible  for  the  validity  coefficient  to 
exceed  the  inter- judge  reliability  coefficient.  Therefore, 
at  least  three  physicians  should  independently  review  the 
charts  of  each  patient  in  the  study,  and  indicate  (using  the 
form  in  Appendix  B-5)  whether  the  patients'  conditions  were 
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in  fact  emergent,  urgent,  or  nonurgent. 


The  accuracy  of  the  staff  in  estimating  the  patient's 
"true  urgency"  can  be  reported  in  four  ways:  (1)  the 

correlation  between  the  staff  urgency  ratings  and  the  "true 
urgency";  (since  the  data  is  ordinal  and  tied  ranks  are 
expected,  the  Kendall  correlation  coefficient  is  most 
appropriate  (Blalock,  1972,  p.  421));  (2)  the  percentage  of 

agreement  between  the  "true  urgency"  and  the  staff 
assessments;  (3)  the  percentage  of  agreement  between  the 
urgency  assessment (s)  made  by  at  least  one  panel  physician 
and  the  staff  assessments;  and  (4)  the  chi  square  goodness 
of  fit  test  (Ferguson,  1971,  pp.  177-182) ;  (this  last  test 
describes  the  agreement  between  the  frequency  of  assignment 
to  each  of  the  three  urgency  groups) . 

Variables  are  often  hypothesized  to  account  for 
differences  in  the  assignment  of  urgency  ratings  by  the 
panel  or  staff  or  the  patient  himself.  Data  for  some  of 
these  variables  are  included  in  the  patient  records  (Figure 
l.k) .  (For  this  study,  the  investigator  coded  the 
information  listed  in  Appendix  B-6) . 

Actual  Design  Employed 

Modifications  to  the  above  optimal  design  were 
necessary  (see  Figures  3  and  4) .  First,  only  two  emergency 
departments  were  available,  and  accordingly  no 
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generalizations  can  safely  be  made  from  a  nonrandom  sample 
of  two  hospitals*.  However,  replicating  the  study  in 
two  hospitals  did  provide  a  crude  measure  of  inter- 
hospital  variation.  Thus,  an  attempt  was  made  to  achieve 
some  degree  of  external  validity  by  replication.  Since  the 
study  was  simply  replicated  a  minimum  of  250  patients  were 
now  required  from  each  hospital.  The  investigator  realized 
that  some  records  would  probably  not  be  located  at  the  time 
of  the  panel  review  for  reasons  which  have  no  apparent 
relationship  to  the  urgency  of  the  patient  (e.g.,  record 
misfiled,  record  temporarily  unavailable  because  patient 
returned  for  cast  change,  follow-up,  etc.).  Accordingly,  the 
sample  size  was  increased  by  10%  to  275  to  ensure  that  at 
least  250  patient  charts  would  be  reviewed  by  each 
hospital's  panel  (Figure  3.g;  Figure  4.g). 

No  pretest  was  conducted  at  either  hospital  due  to 
time  constraints,  although  the  investigator  discussed  the 
data  collection  method  with  staff  members  at  both  hospitals. 

Only  patients  seen  by  emergency  physicians  were 
included  (Figure  3.b;  Figure  4.b).  To  include  those  seen  by 
their  own  doctor  would  have  meant  notifying  all  attending 
staff  and  requesting  their  cooperation  in  familiarizing 
themselves  with  the  urgency  criteria  so  that  they  could 
assign  an  urgency  rating  to  patients  which  they  examined  in 
the  emergency  department.  Persons  in  charge  of  administrating 
the  emergency  department  in  each  hospital  stated  that  it  would 
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a 


An  emergency  physician  reviews  the  criteria  for 
emergent,  urgent,  nonurgent. 


V 


■The  study  sample  is  any  patient  (pt.)  seen  by  an 
emergency  physician  in  the  emergency  department 
(E.D.)  (i.e.,  excludes  patients  seen  by  their 

personal  physician)  during  3  consecutive  weekdays, 
all  shifts. 


V 


c 


Clerk  registering  a  study  patient 

-  rates  urgency  of  patient's  condition 

-  notes  how  long  she  has  worked  in  an  E.D. 

-  marks  pt.  chart  number  on  all  urgency  rating 

forms  and  pt.'s  name  on  clerk's  rating  form  . 


V 


d 


Nurse  accompanying  pt.  to  examination  area 

-  asks  pt.  questions  on  urgency  of  his  condition 

-  rates  urgency  of  patient 

-  notes  how  long  she  has  worked  in  an  E.D. 

-  detaches  her  rating  form  from  pt.  chart. 


V 


e 


Emergency  physician  who  examines  pt. 

-  rates  urgency  of  patient 

-  notes  how  long  he  (she)  has  worked  in  an  E.D. 

-  detaches  physician  rating  form  from  pt.  chart. 


V 


Figure  3:  Modified  design  for  hospital  A 
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f 


Staff  urgency  rating  forms  are  collected  within  24 
hrs.  of  completion  &  checked  for  incomplete  data. 


V 


g 


275  charts  are  randomly  selected  from  charts 
of  all  patients  in  the  study  sample,  excluding 
those  where  a  staff  member  didn't  complete  rating. 


V 


h 


Wait  14  days. 


V 


l 


Charts  of  275  selected  patients  are  retrieved. 
Panel  of  3  physicians  independently  assign  urgency 
ratings  to  charts.  Patients  seen  by  part  time 
physicians  and  charts  where  information  was 
inadequate  were  noted. 


V 


j 


Data  coded  from  patient  charts. 


Figure  3:  Modified  design  for  hospital  A  (continued) 
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a 


Emergency  physicians  review  criteria  for 
emergent,  urgent,  nonurgent. 


V 


b 


The  study  sample  is  any  pt.  seen  by  an  emergency 
physician  during  1  week,  all  shifts. 


V 


c 


Nurse  accompanying  pt.  to  examination  area 

-  asks  pt.  questions  related  to  urgency  of  his 

condition 

-  rates  urgency  of  patient 

-  notes  how  long  she  has  worked  in  an  E.D. 

-  marks  pt.  chart  number  on  nurse  and  physician 

rating  forms 

-  attaches  physician  rating  form  to  pt.  chart. 


V 


d 


Emergency  physician  examining  pt. 

-  rates  urgency  of  patient 

-  notes  how  long  he  (she)  has  worked  in  an  E.D. 

-  detaches  physician  rating  form  from  pt.  chart. 


V 


e 


Staff  urgency  rating  forms  are  collected  at  least 
every  second  day,  coded  &  checked  for  in- 
completions. 


Figure  4:  Modified  design  for  hospital  B 
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Wait  9  days. 


V 


g 


Panel  of  3  physicians  rate  urgency  of  275  pt . 
charts,  randomly  selected  from  all  charts  for 
which  pt.  was  rated  by  nurse  and  physician. 


V 


h 


Data  coded  from  patient  charts. 


Figure  4 : 


Modified  design  for  hospital  B  (continued) 
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be  impossible  to  ensure  the  cooperation  of  these  physicians. 


This  decision  placed  bounds  on  the  physician 
population  under  study  and  created  the  potential  for  another 
type  of  missing  data,  namely,  omissions  due  to  the  clerk's 
belief  that  the  patient  was  to  be  seen  by  his  own  doctor 
when  in  fact  an  emergency  physician  cared  for  him. 

After  discussions  with  the  staff  of  the  emergency 
departments,  it  was  realized  that  by  randomly  choosing  time 
periods  which  were  stratified  by  day  of  the  week,  time  of 
day,  and  time  of  the  year,  the  extent  of  missing  data  would 
be  high  since  it  was  unreasonable  to  expect  the  staff, 
especially  during  busy  periods,  to  commence  and  conclude  the 
sampling  during  the  exact  times  chosen.  Furthermore,  the 
persons  in  charge  of  the  emergency  departments  of  the  two 
hospitals  stated  that  it  would  be  very  difficult  to  maintain 
the  interest  and  cooperation  of  all  the  emergency  staff  for 
even  a  one  month  period,  let  alone  for  a  year. 

A  high  rate  of  missing  data  would  greatly  decrease 
internal  validity  and  "Internal  validity  is  the  basic 
minimum  wi  thout  which  any  experiment  is  uninterpretable 
(Campbell  &  Stanley,  1963,  p.  5)."  Consequently,  external 
validity,  or  generalizability ,  was  sacrificed  to  increase 
internal  validity  by  choosing  consecutive  patients  over  the 
minimum  time  period  necessary  to  include  275  patients  in  the 
sample.  This  decision  increased  the  probability  that  the 
sample  was  not  representative  of  the  actual  emergency 
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department  caseload;  on  the  other  hand  it  also  greatly 
increased  the  chances  that  the  staff  would  adhere  to  and  use 
the  urgency  criteria. 

The  patient  load  in  hospital  A  was  approximately 
twice  that  of  hospital  B.  Consequently,  the  study  ran  for  a 
full  week  in  hospital  A  but  only  for  3  weekdays  in  hospital 
B  (Figure  3,b;  Figure  4.b).  In  both  hospitals,  all  shifts 
were  included  during  the  data  collection  period. 

Hospital  A 

Panel  physician  I  in  hospital  A  (cf.  Ch.  2)  was 
concerned  that  both  full  and  part  time  emergency  physicians 
completed  the  assessments.  Since  he  believed  there  could  be 
variations  here,  he  noted  which  patients  were  seen  by  part- 
time  physicians  when  he  completed  his  assessments.  In 
addition,  all  panel  physicians  (I,  II,  III)  in  this  hospital 
indicated  which  patient  charts  they  felt  had  insufficient 
information  to  allow  them  to  be  confident  in  their 
assessments.  Both  these  items  were  added  to  the  information 
coded  by  the  investigator  (Figure  3.i). 

The  patient  questions  provided  as  examples  in  the 
optimal  design  (cf.  p.  26)  were  used  as  an  indirect  measure 
of  the  patient's  perception  of  his  urgency  in  this  hospital 
(cf .  Appendix  B-2) . 
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The  patients'  chart  in  hospital  A  did  not  include 
marital  status  information,  although  information  was 
collected  on  whether  or  not  the  patient  was  a  Workman's 
Compensation  Board  (WCB)  case. 

Hospital  B 

The  registration  clerk  was  not  included  in  the  study 
at  this  hospital  because  she  was  considered  unqualified  for 
such  an  assessment  by  those  responsible  for  the  service 
being  rendered  in  the  emergency  department. 

Patient  question  2,  cited  on  page  26,  was  not  asked 
because  persons  providing  care  in  this  emergency  department 
felt  that  this  could  have  a  negative  affect  upon  public 
relations.  Therefore,  a  question  relating  to  physician 
contact  prior  to  the  emergency  visit  was  substituted 
(Appendix  B-3) . 

Summary 

An  optimal  design  for  investigating  the  ability  of 
emergency  department  staff  to  classify  the  urgency  of 
patients'  conditions  should  include:  (1)  a  random  selection 

of  emergency  departments  from  all  emergency  departments  of 
interest;  (2)  the  use  of  urgency  criteria  which  have  been 
accepted  in  the  literature;  (3)  a  review  of  these  criteria 
by  persons  in  the  emergency  departments  being  studied;  (4) 
the  selection  of  a  minimum  of  250  patients,  randomized. 
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clustered,  and  stratified  by  appropriate  variables  over  the 
time  period  of  interest;  (5)  the  independent  assignment  of 
urgency  ratings  by  the  patient  and  all  staff  members  of 
interest;  (6)  the  assignment  of  a  "true  urgency"  rating  to 
each  selected  patient,  against  which  the  accuracy  of  the 
staff  and  the  patient  will  be  judged;  (7)  the  collection  of 
data  which  may  account  for  differences  in  the  assignment  of 
urgency  ratings  by  the  staff  and  patient;  and  (8)  a  pretest 
of  the  methods  employed. 

The  investigator  wished  to  study  the  accuracy  of  the 
clerk,  nurse,  and  physician  in  assessing  the  urgency  of 
patients  entering  Edmonton  emergency  departments  during  a 
one  year  period.  The  design  actually  employed  in  this  study 
did  not  fulfill  all  the  criteria  required  for  an  optimal 
design.  The  following  deviations  were  necessary  for 
implementation  of  the  study;  (1)  two  emergency  departments 
were  nonrandomly  selected;  (2)  the  sample  for  each  hospital 
was  consecutive  patients  seen  by  only  emergency  physicians, 
during  no  more  than  seven  consecutive  days  in  June  1973;  (3) 

the  clerk  was  not  included  in  the  study  at  one  hospital;  all 
three  types  of  staff  were  included  in  the  second  hospital; 
and  (4)  the  questions  prepared  to  provide  an  indirect 
measure  of  the  patient's  perception  of  the  urgency  of  his 
condition  had  to  be  modified  for  one  of  the  hospitals. 


An  explanation  of  the  results  now  follows 
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CHAPTER  FOUR 


ANALYSIS  AND  INTERPRETATION 
Introduction 

This  chapter  reports  the  findings  of  the 
implementation  of  the  research  design  described  in  the 
preceding  chapter.  The  results  and  interpretation  of  the 
data  analysis  is  presented  in  the  following  order:  (1) 

report  of  missing  data,  (2)  analysis  of  urgency  ratings 
assigned  by  the  panel  physicians,  (3)  analysis  of  the  urgency 
ratings  assinged  by  each  type  of  staff,  (4)  effect  of 
experience  on  the  ability  of  staff  to  rate  the  urgency  of 
patients,  (5)  the  patient's  perception  of  his  urgency,  and 
(6)  other  factors  influencing  the  accuracy  of  the  staff 
urgency  ratings. 

Report  of  Missing  Data 

Although  steps  were  taken  to  eliminate  sources  of 
missing  data  wherever  possible  (cf .  p.  30-35) ,  this  problem 
was  not  entirely  overcome.  Table  1  summarizes  the  extent  of 
incomplete  data  collection  in  both  hospitals. 

During  the  data  collection  period  at  least  401 
patients  from  hospital  A  and  464  patients  from  hospital  B 
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TABLE  1 


EXTENT  OF  INCOMPLETE  DATA  COLLECTION 


Hospital  A  Hospital  B 


i  -  Minimum  number  of 

patients  seen  by  emergency  401  (100%) 

physicians  during  data 
collection  period. 

ii  -  Total  number  of  patients 

assessed  by  a  clerk,  and/or  a  401  (100%) 

nurse,  and/or  an  emergency 

physician. 

iii  -  Number  of  patients 

given  a  rating  by  both  359  (  90%) 

physician  and  nurse 

iv  -  Number  of  records 

randomly  selected  from  iii  275  (  69%) 

for  review  by  the  panel. 

v  -  Number  of  records  assessed 
by  all  3  panel  physicians 
(i.e.,  iv  less  those  charts 
that  could  not  be  located  at 
the  time  of  the  review) . 

vi  -  Number  of  records 
investigator  was  able  to 
assign  a  "true  urgency" 
rating  (cf .  pp. 43-44)  . 

vii  -  Number  of  records  rated 
by  the  clerk  which  had  been 
given  a  "true  urgency"  rating 


252  (  63%) 

250  (  62%) 

249  (  62%) 


464  (100%) 

431  (  93%) 

368  (  79%) 

275  (  59%) 

266  (  57%) 

265  (  57%) 
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(Table  l.i)  were  seen  by  emergency  physicians.  The 
investigator  had  no  way  of  accurately  determining  the  exact 
number  of  patients  seen  by  emergency  physicians  during  the 
study  period.  In  hospital  A,  the  charts  of  patients  seen 
during  the  study  period  were  filed  with  those  of  previous 
months.  Thus  it  was  virtually  impossible  to  determine  how 
many  patients  were  seen  by  an  emergency  physician  during 
the  study  period,  but  were  not  assigned  ratings  by  any  staff 
member.  This  number  was  not  believed  to  be  large  since 
extra  clerical  support  was  available  to  ensure  that  the 
urgency  rating  forms  were  attached  to  the  charts  of  all 
patients  who  were  to  be  seen  by  an  emergency  physician.  On 
the  other  hand,  the  emergency  department  of  hospital  B  was 
understaffed  on  days  5  and  6  of  the  study.  Consequently, 
the  staff  was  unable  to  formulate  urgency  assessments  for 
some  patients.  There  were  also  short  periods  of  time  during 
other  study  days  when  the  staff  were  very  busy  and  did  not 
have  time  to  assign  ratings.  During  the  study  in  hospital  B, 
the  charts  of  all  patients  seen  in  this  emergency  department 
were  filed  separately  from  the  rest  of  the  1973  patient 
charts.  By  reviewing  the  charts  of  all  patients  seen  during 
the  study  week,  the  investigator  was  able  to  identify  33 
patients  that  were  seen  by  an  emergency  physician  but  not 
rated  by  the  staff.  Thus,  at  least  seven  percent  of  the 
patients  selected  for  this  study  were  not  rated  by  the  staff 
(Table  l.ii) .  These  patients  did  not  appear  to  differ  from 
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the  study  patients  in  the  type  of  illness  or  injury  which 
brought  them  to  the  emergency  department,  but  many  of  them 
were  seen  by  more  than  one  doctor  due  to  a  shift  change,  or 
the  need  for  consultation.  A  few  of  these  patients  were  seen 
by  both  the  emergency  physician  and  their  own  doctor. 

Since  the  staff  in  both  hospitals  were  very  busy 
during  the  study,  there  were  times  when  the  nurse  or  doctor 
did  not  complete  the  appropriate  rating  form.  For  this 
reason,  42  cases  (10%)  were  incomplete  in  hospital  A  and  63 
caseis  (14%)  in  hospital  B.  These  patients  did  not  appear  to 
differ  in  the  type  of  illness  or  injury  from  those  assessed 
by  all  types  of  staff.  Some  of  these  cases  were  patients  who 
were  seen  by  their  own  doctor  but  the  staff  member  attaching 
the  forms  believed  they  were  to  be  seen  by  an  emergency 
physician. 

The  number  of  patients  rated  by  the  three  types  of 
staff  in  each  hospital  was  considerably  greater  than  275 
(the  number  of  patient  charts  the  panel  had  agreed  to  review 
for  this  study) .  Therefore,  the  number  of  charts  selected 
for  review  by  the  panel  was  reduced  to  275  by  random  sampling 
(Table  l.iv) . 

Some  records  could  not  be  located  at  the  time  of  the 
panel  review.  In  hospital  A,  the  charts  of  the  patients 
rated  by  the  staff  were  retrieved  by  patient  name  from  the 
charts  of  all  patients  seen  in  previous  months.  The 
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investigator  was  unable  to  locate  23  of  the  275  charts  since 
some  patient  names  were  not  written  clearly  on  the  clerk's 
rating  form  (Appendix  B-l)  ,  and  the  investigator  believes 
that  some  charts  may  have  been  misfiled.  Therefore  only  252 
charts  were  reviewed  by  the  panel  in  hospital  A  (Table  1 
v)  . 

By  the  time  of  the  panel  review  in  hospital  B,  four 
records  for  the  last  study  day  had  not  been  filed  with  the 
records  for  the  study  week.  The  investigator  could  not 
locate  another  five  records  for  patients  seen  during  the 
other  study  days.  Consequently,  only  266  of  the  275  randomly 
selected  records  were  reviewed  by  the  panel.  Only  record 
numbers  (not  patient  names)  were  recorded  during  the  study  in 
this  hospital.  Therefore,  the  investigator  could  not  determine 
if  these  records  had  accidently  been  filed  with  the  rest  of 
the  1973  records.  No  other  reason  is  apparent  for  not  being 
able  to  locate  these  records,  since  any  records  removed  from 
the  charts  of  the  patients  seen  during  the  study  week  were 
signed  out  so  that  they  could  be  located  at  the  time  of  the 
panel  review „ 

In  each  hospital,  there  was  one  patient  for  which 
each  of  the  three  panel  members  assigned  a  different  urgency 
rating.  In  hospital  A,  the  patient  complained  of  chest  pain 
but  the  cause  could  not  be  determined  at  the  time  of  the 
visit.  In  hospital  B,  the  patient  had  taken  an  overdose  of 
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methylenedioxy  amphetamine  (MDA)  and  valium.  Since  these 
two  patients  could  not  be  assigned  a  "true  urgency"  rating 
(i.e.,  a  rating  assigned  by  at  least  two  panel  physicians), 
they  were  omitted  from  the  analysis.  Another  patient  from 
hospital  A  was  also  omitted  since  he  was  only  rated  by  two 
of  the  three  panel  physicians.  Although  the  other  two 
physicians  rated  this  patient  as  urgent,  the  investigator 
believed  that  if  any  physician  declined  to  rate  a  patient 
because  of  insufficient  information,  the  validity  of  the 
rating  would  be  in  doubt,  and  thus  should  not  be  declared 
the  "true  urgency"  against  which  the  accuracy  of  staff  members 
was  to  be  judged.  There  were  no  cases  where  two  physicians 
listed  a  case  as  (1)  nonurgent  and  the  third  physician  placed 

him  in  the  emergent  category,  or  (2)  emergent  and  the  third 
physician  placed  him  in  the  nonurgent  category. 

The  sample  sizes  used  in  reporting  the  accuracy  of  the 
nurse  and  emergency  physician  are  250  for  hospital  A  and  265 
for  hospital  B  (Table  l.vi) .  The  sample  size  employed  in 
reporting  the  accuracy  of  the  clerk  in  hospital  A  was  249 
(Table  1.  vii)  since  one  patient,  who  was  rated  by  the  nurse 
and  emergency  physician,  was  not  rated  by  the  clerk. 

Analysis  of  Urgency  Ratings  Assigned  by  the  Panel  Physicians 

"True  urgency"  ratings  were  formulated  from  the 
"panel"  physician  ratings  (cf .  p.  28) .  The  validity  of  these 
ratings  is  unknown,  but  theoretically  cannot  be  greater  than 
the  reliability  of  these  ratings.  Reliability  estimates 
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between  panel  physicians ' are  included  in  Table  2.  Kendall 
correlation  coefficients  between  pairs  of  panel  physicians 
in  each  hospital  ranged  from  0.449  to  0.666.  Corresponding 
percentages  of  agreement  ranged  from  65%  to  82%.  A 
breakdown  of  the  percentages  of  agreement  (Table  3)  shows 
that  only  one  patient  in  each  hospital  was  assigned  an 
"emergent"  rating  by  all  three  physicians.  Slightly  more  than 
a  quarter  of  the  patients  were  classified  as  "urgent"  by  all 
panel  members  (33.1%  in  hospital  A,  27.5%  in  hospital  B.) 
Approximately  another  quarter  were  labelled  "nonurgent"  by 
all  three  panel  physicians  (25.9%  in  hospital  A,  22.3%  in 
hospital  B) .  The  three  physicians  did  not  agree  unanimously 
in  over  40%  of  the  ratings  (40.2%  for  hospital  A,  49.8%  for 
hospital  B) . 

Since  unanimity  amongst  panel  members  was  lacking  in 
over  40%  of  the  cases,  serious  doubts  are  raised  as  to  the 
validity  of  using  the  review  of  records  by  a  single  physician 
as  an  accurate  measure  of  patient  urgency,  both  in  this  study 
and  in  studies  by  other  investigators  (cf .  p.  12) .  To 
investigate  the  extent  of  the  discrepancies  amongst  the  panel 
physicians,  Friedman's  two-way  analysis  of  variance  by  ranks 
(Siegal,  1956,  pp.  166-172)  was  carried  out  to  test  the  null 
hypothesis  that  there  was  no  significant  difference  between 
the  ratings  of  the  panel  physicians.  The  chi  square  values 
obtained  had  probabilities  less  than  0.001  (chi  square  for 
hosptial  A  was  67.8,  df=2;  for  hospital  B  was  35.19,  df=2) . 
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TABLE  2 

PERCENTAGES  OF  AGREEMENT  AND  KENDALL  CORRELATION  COEFFICIENTS 
FOR  URGENCY  RATINGS  ASSIGNED  BY  THE  STAFF  AND  PANEL* * 


Hospital  A  (N=250  ° ) 


Clerk 

Nurse 

Doctor 

I 

II 

III 

True 

urgency 

Clerk 

1.000 

64% 

57% 

61% 

59% 

62% 

62% 

Nurse 

0.443 

1.000 

73% 

72% 

63% 

64% 

70% 

Doctor 

0.351 

0.527 

1.000 

71% 

71% 

62% 

70% 

I 

0.423 

0.501 

0.501 

1.000 

82% 

73% 

95%** 

II 

0.387 

0.342 

0.505 

0.666 

1.000 

65% 

87%** 

III 

0.376 

0.441 

0.494 

0.593 

0.465 

1.000 

78%** 

True 

Urgency 

0.434 

0.463 

0.494 

0.910** 

0.770** 

0.659 

**  1.000 

Note.-  Percentages  of  agreement  values  are  above  the 
diagonal  and  Kendall  correlation  coefficients  below.  Kendall 
correlation  coefficients  have  smaller  absolute  values  (by 
approximately  0.01  in  these  tables)  than  Spearman  rho 
correlation  coefficients.  The  two  coefficients  provide 
identical  information ,  except  that  Kendall's  tau  tends  to  be 
more  meaningful  when  many  tied  ranks  are  present  in  the  data 
(Blalock,  1972,  p.  421). 

*  Roman  numerals  I,  II,  and  III  denote  the  three  panel 
physicians  in  hospital  A,  while  IV,  V,  and  VI  indicate  the 
panel  physicians  in  hospital  B. 

c  Except  where  the  measures  of  agreement  include  the 
clerk's  ratings,  in  which  case  N  =  249. 

**  These  values  are  inflated  since  they  are  relationships 
between  a  panel  physician's  rating  and  a  panel  composite 
score . 
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TABLE  2 
(CONTINUED) 

PERCENTAGES  OF  AGREEMENT  AND  KENDALL  CORRELATION  COEFFICIENTS 
FOR  URGENCY  RATINGS  ASSIGNED  BY  THE  STAFF  AND  PANEL* 


Hospital 

B  (N= 

Clerk 

265) 

Nurse 

Doctor 

IV 

V 

VI 

True 

urgency 

Nurse 

— 

— 

1.000 

69% 

66% 

64% 

64% 

66% 

Doctor 

0.448 

1.000 

66% 

59% 

70% 

66% 

IV 

0.387 

0.363 

1.000 

67% 

67% 

84%** 

V 

0.372 

0.335 

0.449 

1.000 

66% 

79%** 

VI 

0.328 

0.418 

0.487 

0.547 

1.000 

84%** 

True 

urgency 

0.350 

0.403 

0.707**  0.706**  0.712** 

1.000 

*  Roman  numerals  I,  II,  and  III  denote  the  three  panel 
physicians  in  hospital  A,  while  IV,  V,  and  VI  indicate  the 
panel  physicians  in  hospital  B. 

**  These  values  are  inflated  since  they  are  relationships 
between  a  panel  physician's  rating  and  a  panel  composite 
score . 
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TABLE  3 

FREQUENCY  OF  VARIOUS  COMBINATIONS  OF  PANFL  PHYSICIAN  URGENCY 

RATINGS 


Hospital 

A 


Hospital 

B 


True 

urgency 


emergent,  emergent, 
emergent 

emergent,  emergent, 
urgent 

emergent,  urgent, 
urgent 

urgent,  urgent, 
urgent 

urgent,  urgent, 
nonurgent 

urgent,  nonurgent, 
nonurgent 


1  (  0.4%) 

3  (  1.2%) 

11  (  4.4%) 


39  (  15.5%) 
48  (  19.1%) 


nonurgent,  nonurgent, 

nonurgent  65  (  25.9%)* 


1  (  0.4%)*  emergent 


3  (  1.1%)  emergent 


3  (  1.1%)  urgent 


83  (  33.1%)*  73  (  27.5%)*  urgent 


58  (  21.9%)  urgent 


68  (  25.7%)  nonurgent 


59  (  22.3%)*  nonurgent 


Total 


250  (100.0%)  265  (100.0%) 


*  The  sum  of  these  values  for  each  hospital  (59.8% 
for  hospital  A;  50.2%  for  hospital  B)  is  the  proportion  of 
cases  where  all  three  panel  physicians  were  in  agreement. 
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Therefore,  one  would  reject  the  null  hypthesis,  i.e.,  there 
were  significant  differences  between  panel  physicians. 
Consequently,  for  purposes  of  this  study,  the  urgency  rating 
given  by  at  least  two  of  the  panel  physicians  was  coded  as 
the  "true  urgency"  for  each  patient.  While  this  "true 
urgency"  rating  contains  an  unknown  error  component,  it  was 
considered  the  best  estimate  of  the  actual  urgency  of  the 
patient's  condition  available  to  the  investigator.  (Recall 
that  only  one  patient  in  each  hospital  was  assigned  different 
ratings  by  all  three  panel  physicians;  these  two  patients 
were  omitted  from  this  analysis  (cf .  p.  44) .) 

Analysis  of  Urgency  Ratings  Assigned  by  the  Staff 

For  the  discussion  and  tables  in  this  chapter,  "clerk" 
refers  to  all  clerical  personnel  registering  patients  in  the 
emergency  department  being  considered,  "nurse"  to  all  nurses 
providing  nursing  care  in  the  emergency  department  of 

I  '! 

hospital  A  or  B,  and  "doctor"  to  the  full  and  part-time 
emergency  physicians  responsible  for  medical  coverage  of  the 
emergency  department  under  study.  "Staff"  is  used  to  include 
only  these  three  types  of  personnel. 

The  Kendall  correlation  coefficients  and  percentages 
of  agreement  between  the  staff  urgency  ratings  and  the  "true 
urgency"  ratings  are  provided  in  Table  2.  The  range  in 
correlation  coefficients  was  from  0.350  to  0.494.  While 
none  of  the  correlation  coefficients  between  the  staff  and 
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"true  urgency"  ratings  can  be  considered  high,  they  are  all 
statistically  significant  under  the  hypothesis  that  tau 
equals  zero. 

The  percentages  of  agreement  between  the  assessments 
of  the  three  types  of  staff  and  the  "true  urgency"  ratings 
(Table  2)  show  that  the  nurse  and  the  doctor  were  equally 
capable.  Both  classified  70%  of  the  patients  correctly  in 
hospital  A  and  66%  in  hospital  B.  The  clerk's  rating  agreed 
with  the  "true  urgency"  ratings  62%  of  the  time.  Using  a 
two-tailed  proportions  test  with  a  level  of  significance  of 
0.05,  the  clerk  was  found  to  be  significantly  less  accurate 
than  the  doctor  (z  =  1.99,  p  <  0.05)  but  not  significantly 
less  accurate  than  the  nurse  (z  =  1.95,  p  >  0.05).  Note  that 
both  z  values  were  close  to  the  critical  z  value  of  1.96. 

The  differences  arose  from  the  disparity  in  the  nurse  and 
physician  ratings  before  rounding  (nurse  =  69.6%,  doctor  = 
70.4%) . 

As  stated  previously  (cf.  p.  49),  the  panel  members 
did  not  agree  unanimously  on  the  urgency  ratings  for  over 
40%  of  the  patients.  Consequently,  the  number  of  times  the 
ratings  agreed  with  at  least  one  of  the  panel  physicians  is 
also  provided  (Table  4).  The  physician's  agreement  was  only 
marginally  greater  than  the  nurse's  agreement.  The  nurse  in 
hospital  A  agreed  with  at  least  one  of  the  panel  physicians 
85.6%  of  the  time,  while  the  doctor  was  in  agreement  86.8% 
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TABLE  4 

NUMBER  OF  CASES  WHERE  THE  STAFF  RATINGS  AGREED  WITH  AT  LEAST 

ONE  OF  THE  PANEL  RATINGS 


Hospital  A 

Hospital  B 

Clerk  200  (  80.3%) 

(N  =  249) 

Nurse  214  (  85.6%) 

(N  =  250) 

234  (  88.3%) 

(N  =  265) 

Doctor  217  (  86.8%) 

(N  =  250) 

237  (  89.4%) 

(N  =  265) 
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of  the  time.  In  hospital  B,  these  values  were  88.3%  for  the 
nurse  and  89.4%  for  the  doctor.  In  hospital  A,  the  clerk's 
agreement  (80.3%)  was  significantly  lower  than  the  nurse  (z 
=  2.00,  p  <  0.05)  and  the  doctor  (z  =  2.27,  p  <  0.05). 

The  chi  square  goodness  of  fit  test  (Ferguson,  1971, 
pp.  171-182)  was  used  to  determine  v/hich  staff  members  were 
most  accurate  in  estimating  the  proportions  of  patients 
falling  into  the  nonurgent,  urgent,  and  emergent  categories. 
(Criterion  for  these  "true  urgency"  categories  is  given  in 
Table  3.)  Table  5  includes  the  results  of  this  test,  and  the 
percentage  of  patients  rated  as  "emergent",  "urgent",  and 
"nonurgent"  by  each  staff  member  and  panel  physician.  Using 
a  level  of  significance  of  0.05,  one  would  conclude  that  the 
clerk,  the  emergency  physician  in  both  hospitals,  and  panel 
physicians  III,  V,  and  VI  did  not  accurately  estimate  the 
"true"  proportion  of  patients  in  each  urgency  category. 

Since  three  of  the  six  panel  physicians  are  assumed  to  be 
unable  to  make  such  an  estimate,  one  must  question  (1)  the 
reported  proportions  of  "true"  emergent,  urgent,  and 
nonurgent  patients  in  this  study  (cf.  Table  5  "True  urgency") , 
and  in  other  studies  (cf .  p.  12) ,  and/or  (2)  the  advisability  of 
using  a  review  of  patient  records  for  establishing  "true 
urgency",  and/or  (3)  the  criteria  establishing  "true  urgency" 
(i.e.,  rating  agreed  upon  by  at  least  two  physicians, 
definitions  and  examples  of  emergent,  urgent,  and  nonurgent 
(Appendix  A)  )  . 
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TABLE  5 

PERCENT  EMERGENT,  URGENT,  AND  NONURGENT  BY  PERSON  MAKING 
ASSESSMENT  AND  TEST  FOR  CHI  SQUARE  GOODNESS  OF  FIT 

X^Test  for 

Emergent  Urgent  Nonurgent  Goodness  of 

Fit* (df =2) 


Hospital 

A 

Clerk 

24 

(9.6%) 

140 

(56.2%) 

8  5 

(34.2%) 

104.52 

Nurse 

7 

(2.8%) 

120 

(48.0%) 

123 

(49.2%) 

3.79 

Doctor 

10 

(4.0%) 

126 

(50.4%) 

114 

(45.6%) 

8.00 

I 

4 

(1.6%) 

126 

(50.4%) 

120 

(48.0%) 

1.22 

II 

2 

(0.8%) 

119 

(47.6%) 

129 

(51.6%) 

5.26 

III 

14 

(5.6%) 

157 

(62.8%) 

79 

(31.6%) 

58.67 

True  urgency  4 

(1.6%) 

134 

(53.6%) 

112 

(44.8%) 

Hospital 

B 

Nurse 

S' 

(2.3%) 

131 

(49.4%) 

128 

(48.3%) 

1.08 

Doctor 

5 

(1.9%) 

109 

(41.1%) 

151 

(57.0%) 

8.37 

IV 

3 

(1.1%) 

142 

(53.6%) 

120 

(45.3%) 

1.11 

V 

6 

(2.6%) 

170 

(63.9%) 

89 

(33.5%) 

22.04 

VI 

3 

(1.1%) 

100 

(37.8%) 

162 

(61.1%) 

18.53 

True  urgency  4 

(1.5%) 

134 

(50.6%) 

127 

(47.9%) 

*  The  chi  square  goodness  of  fit  test  (Ferguson,  1971, 
p.  177-182)  was  used  to  compare  the  number  of  patients  that 
the  staff  and  panel  placed  in  each  urgency  group  to  the  number 
of  patients  in  each  group  of  the  "true  urgency"  rating. 
Underlined  values  are  those  where  the  number  of  patients  placed 
in  each  group  was  not  related  to  the  final  urgency  distribution, 

when  the  level  of  significance  is  set  at  0.05. 
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With  these  cautions  in  mind,  the  results  of  Table  5 
will  now  be  discussed.  In  both  hospitals,  the  nurse  assigned 
the  same  proportion  of  patients,  but  not  necessarily  the  same 
patients  to  each  urgency  category  as  found  in  the  "true 
urgency"  groups  (hospital  A,  chi  square  =  3.79,  df  =  2,  p  > 
0.05;  hospital  B,  chi  square  =  1.08,  df  =  2,  p  >  0.05).  The 
emergency  physician  did  not  assign  the  same  proportions  as  the 
"true  urgency"  ratings  (hospital  A,  chi  square  =  8.00,  df  =  2, 
p  <  0.05;  hospital  B.  chi  square  =  8.37,  df  =  2,  p  <  0.05). 

The  registration  clerk  did  not  include  the  same 
proportion  of  patients  in  each  urgency  category  as  given  in 
the  "true  urgency"  rating  (chi  square  =  104.52,  df  =  2 ,  p  < 
0.05) .  She  tended  to  view  the  patients  as  more  urgent  than 
reported  by  the  "true  urgency"  ratings  (Table  6) .  This 
conservative  approach  may  have  merit  in  a  triage  situation 
but  must  be  weighted  against  the  decreased  effectiveness 
of  a  triage  system  in  which  the  nurse  would  be  responsible 
for  streaming. 

Table  7  provides  further  evidence  that  the  clerk 
tended  to  be  more  conservative  in  her  assessments  of  urgency. 

She  was  the  only  staff  member,  in  either  hospital,  to  label 

♦ 

"emergent"  all  four  patients  whose  "true  urgency"  was  emergent 
Also,  the  clerk  had  almost  10%  fewer  correct  nonurgent  ratings 
than  the  other  staff  members  in  both  hospitals. 
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TABLE  6 

CASES  OF  DISAGREEMENT  BETWEEN  STAFF  AND  TRUE  URGENCY 

ASSESSMENTS 


Assessed  higher  Assessed  lower  Classified 

than  panel  than  panel*  incorrectly 


Hospital  A 

Clerk 

70 

( 

28.1%) 

24 

( 

9.6%) 

94 

( 

37.7%) 

Nurse 

34 

( 

13.6%) 

42 

( 

16.8%) 

76 

( 

30.4%) 

Doctor 

39 

( 

15.6%) 

35 

( 

14.0%) 

74 

( 

29.6%) 

Hospital  B 

Nurse 

44 

( 

16.6%) 

46 

( 

17.3%) 

90 

( 

33.9%) 

Doctor 

33 

( 

12.4%) 

56 

( 

21.1%) 

89 

( 

33.5%) 

*  No  staff  member  ever  classified  a  patient  as 
nonurgent  that  the  panel  later  listed  as  emergent. 

TABLE  7 

CASES  OF  AGREEMENT  BETWEEN  STAFF  AND  TRUE  URGENCY 
ASSESSMENTS  FOR  THE  THREE  URGENCY  CATEGORIES 


Classified 

Emergent  Urgent  Nonurgent  correctly 


Hospital 

A 

Clerk 

4 

( 

1.6%) 

90 

( 

36.1%) 

61 

( 

24.5%) 

155 

( 

62.2%) 

Nurse 

2 

( 

0.8%) 

89 

( 

35.6%) 

83 

( 

33.2%) 

174 

( 

69.6%) 

Doctor 

2 

( 

0.8%) 

93 

( 

37.2%) 

81 

( 

32.4%) 

176 

( 

70.4%) 

Hospital 

B 

Nurse 

1 

( 

0.4%) 

89 

( 

33.6%) 

85 

( 

32.1%) 

175 

( 

66.1%) 

Doctor 

1 

( 

0.4%) 

77 

( 

29.1%) 

98 

( 

37.0%) 

176 

( 

66.5%) 

Note.-  Denominator  of 

percentages  is 

the 

number  of 

cases 

assessed  by  each  staff  member. 
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The  dependent  proportions  test  (Ferguson,  1971,  pp.  162- 
164)  was  used  to  determine  if  there  was  a  significant  difference 
between  types  of  staff  in  terms  of  the  tendency  to  assess  patients 
in  a  category  of  greater  urgency  than  the  panel's  "true  urgency" 
rating  (i.e.,  staff  ratings  were  dichotomized  into  (1)  ratings 
of  greater  urgency  than  the  "true  urgency"  assessment,  or  (2) 
ratings  the  same  as  or  of  lesser  urgency  than  the  "true  urgency" 
ratings) .  No  statistically  significant  difference  was  observed 
when  the  nurse  and  physician  in  hospital  A  were  compared  (z  = 

0.74,  p  >  0.05).  However,  the  clerk  classified  significantly 
more  patients  in  categories  of  greater  urgency  than  the  nurse 
(z  =  4.78,  p  <  0.05)  or  the  doctor  (z  =  3.68,  p  <  0.05)  .  Cases 
of  incorrect  assessment  (Table  6)  were  investigated  further  to 
identify  cases  of  extreme  divergence  of  opinion  between  staff  and 
"true  urgency"  ratings  (i.e.,  nonurgent  and  emergent  differences). 
In  this  study,  no  staff  member  ever  classified  a  "true  emergent" 
as  nonurgent.  The  nurse  in  hospital  B  rated  three  "nonurgent" 
patients  as  emergent,  and  the  clerk  in  hospital  A  rated  one 
"nonurgent"  patient  as  emergent.  No  other  extreme  incorrect 
classifications  were  made  in  either  hospital. 

The  differences  between  staff  members  in  ability  to 
assess  urgency  can  be  summarized  as  follows.  A  statistically 
significant  relationship  exists  between  the  ratings  of  all 
three  types  of  staff  and  the  "true  urgency"  ratings.  No 
differences  in  percentages  of  agreement  were  found  between 
the  nurse  and  emergency  physician  in  either  hospital.  It  is 
left  to  the  discretion  of  the  reader  to  decide  whether  the 
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percentage  of  agreement  between  the  clerk  and  "true  urgency" 
ratings  was  meaningfully  lower  than  that  for  the  nurse  and 
emergency  physician.  The  percentages  of  agreement  between 
the  ratings  given  by  each  type  of  staff  and  at  least  one 
panel  physician  showed  that,  statistically,  the  clerk  was 

less  accurate  than  the  nurse  and  doctor.  Therefore,  this 
study  demonstrated  no  statistical  difference  between  the  nurse 

and  doctor  in  ability  to  classify  patients.  The  ability  of  the 
clerk  approached  thdt  of  the  health,  professionals,  although  she 
was  more  conservative  in  her  assessments  of  urgency. 

The  decision  on  which  staff  member(s),  if  any,  have 
sufficient  accuracy  to  be  responsible  for  classifying 
patients,  is  not  answered  since  additional  criteria  is  needed 

to  determine  such  an  answer. 

Effect  of  Experience  on  Ability  of  Staff  to  Rate  Urgency 

The  purpose  of  this  study  was  to  examine  the  effect 
of  training  and  experience  on  the  ability  of  emergency 
department  staff  to  clsssify  the  urgency  of  patients  (cf.  p. 

1) .  A  report  of  the  effect  of  training  was  just  provided. 

Now  the  relationship  observed  between  staff  experience  and 
ability  to  classify  patients  will  be  discussed. 

From  the  data  collected  in  this  study,  experience  did 
not  appear  to  account  for  differences  in  the  ability  to 
assess  urgency.  The  urgency  ratings  assigned  by  the  staff 
in  each  of  four  experience  categories  (cf.  p.  24)  were 
compared  using  Kendall's  tau  (Table  8).  When  these 
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TABLE  8 


MEASURES  OF  ACCURACY  BETWEEN  THE  VARIOUS  STAFF  EXPERIENCE 
COMBINATIONS  AND  THE  TRUE  URGENCY  RATINGS  FOR  HOSPITALS  A 

AND  B  COMBINED 


<  3  mo. 

3  mo.-l  yr. 

1-2  yr. 

2  or  more  yr 

KENDALL 

CORRELATION 

COEFFICIENTS 

Clerk 
(N=249 ) 

0.244 
(N=53 ) 

0.596 

(N=61) 

0.630 
(N=ll ) 

0.426 
(N=116 ) 

Nurse 
(N=515 ) 

0.365 
(N=56 ) 

0.503 

(N=45) 

0.167 

(N=71) 

0.460 

(N=335) 

Doctor 

(N=515) 

0.514 

(N=17) 

0.419 
(N=85 ) 

0.562 

(N=237) 

0.278 

(N=171) 

PERCENTAGES 
OF  ACCURATE 
ASSESSMENTS 

Clerk 

59% 

69% 

55% 

69% 

Nurse 

66% 

71% 

56% 

70% 

Doctor 

77% 

66% 

50% 

60% 

CHI  SQUARE 
GOODNESS 

OF  FIT 

Clerk 

0.08 

16.90 

1.14 

116.30 

Nurse 

9.64 

4.02 

10.03 

2.65 

Doctor 

0.00 

2.54 

8.17 

3.64 

Note.-  All  correlations  are  significant  at  the  0.05 
alpha  level. 

Note.-  The  N's  for  the  percentage  of  agreement  and 
chi  square  tabulations  were  the  same  as  those  specified  in 
the  respective  cells  of  the  correlation  calculations. 
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correlations  were  ranked  and  compared  to  the  length  of 
experience  rank  values  using  Spearman's  rho  correlation 
coefficients  (Blalock,  1972,  pp.  416-418)  ,  experience  was 
found  to  be  not  significantly  related  (alpha  =  0.05) 
to  the  ability  to  classify  the  urgency  of  patients  (clerk, 
rho  =  0.80,  p  =  0.095;  nurse,  rho  =  0.40,  p  =  0.245;  doctor, 
rho  =  0.40,  p  =  0.245).  The  percentages  of  accurate 
assignments  and  the  chi  square  goodness  of  fit  test  to  the 
"true  urgency"  ratings  support  the  conclusion  that  no 
relationship  was  evident  between  the  ability  to  classify 
patients  and  experience. 

Factors  other  than  experience,  or  in  combination  with 
experience,  may  account  for  differences  in  the  ability  of 
staff  members  to  assess  urgency,  but  the  results  of  this 
study  do  not  support  the  hypothesis  that  experience  in  an 
emergency  department  alone  influences  accuracy.  It  is 
possible  that  the  operational  definition  of  experience  in 
this  study  was  not  the  most  appropriate  one  that  could  have 
been  employed.  Beloff  (1968)  contends  that  triage  officers 
should  be  "...  experienced  in  dealing  with  patients  ....  [p. 

68l"/  and  thus  does  not  limit  experience  to  that  obtained  in 
an  emergency  department. 

Another  possible  source  of  error  is  that  staff  who 
were  experienced  in  the  emergency  area  and  capable  of 
accurately  classifying  patients  might  not  have  been  motivated 
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to  conscientiously  apply  the  urgency  criteria  prepared  for 
this  study.  Other  characteristics  cited  in  the  literature 
such  as  maturity  (Beloff,  1968,  p.  68),  responsibility 
(Taubenhauts  &  Simon,  1973,  p.  43),  and  interest  (Beloff, 

1968,  p.  69)  may  be  responsible  for  variations  in  ability  to 
assess  urgency . 

Some  of  the  emergency  physicians  were  only  working 
part-time  in  the  emergency  department.  Therefore,  their 
emergency  department  experience  was  not  as  extensive  as  that 
for  full-time  physicians  with  a  comparable  length  of 
experience.  Also,  those  working  part-time  may  not  have  had 
the  same  degree  of  interest  in  their  work  as  those  working 
full-time.  Consequently,  an  attempt  was  made  to  determine  if 
there  was  a  difference  between  full  and  part-time  physicians 
in  ability  to  assess  urgency.  In  hospital  A  panel  physician 
I  noted  which  patients  had  been  seen  by  part-time  emergency 
physicians  when  he  completed  his  urgency  ratings  (cf .  p.  36) . 
Using  this  information,  a  comparison  was  made  between  the 
abilities  of  full  and  part-time  emergency  physicians  in 
hospital  A.  The  ratings  of  the  full-time  emergency 
physicians  agreed  with  the  "true  urgency"  ratings  71.7%  of 
the  time  (N  =  184),  while  the  part-time  physicians  agreed 
with  the  "true  urgency"  ratings  66.6%  of  the  time  (N  =  66). 
This  difference  was  not  significant  at  an  alph  level  of  0.05 
when  the  independent  proportions  test  (Ferguson,  1971,  pp. 
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160-162,)  was  used  (z  =  1.19,  p  >  0.05).  Thus,  in  this  study, 
no  relationship  was  observed  between  ability  to  classify 
urgency  and  (1)  length  of  experience  in  an  emergency 
department,  or  (2)  whether  the  physician  was  working  full  or 
part  time  in  the  emergency  department. 

Patient's  Perception  of  Urgency 

In  examining  who  was  capable  of  assessing  urgency,  the 
investigator  considered  it  expedient  to  include  the 
perceptions  of  the  patient  himself  (cf .  p.  15) .  Therefore, 
the  nurse  asked  the  patient  questions  (Appendices  B-2  &  B-3) 
dealing  with  specific  urgency  criteria.  From  these  questions 
an  indirect  measure  of  the  patient's  perception  of  his 
urgency  was  obtained. 

Seven  patients  from  hospital  A  and  four  patients  from 
hospital  B  did  not  answer  one  or  both  of  the  patient 
questions.  Five  of  these  patients  were  nonurgent  and  the 
rest  urgent.  Where  the  chi  square  test  of  independence  was 
used  to  analyse  contingency  tables,  the  emergent  and  urgent 
categories  were  collapsed  into  one  category  to  eliminate 
small  or  zero  cell  sizes. 

Who  Answered  the  Questions 

The  nurse  was  asked  to  indicate  whether  the  patient 
answered  these  questions  himself  or  someone  accompanying  the 
patient  answered  them.  In  each  hospital,  approximately  30% 
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of  the  responses  were  made  by  the  patient's  companion. 

Despite  this,  no  significant  relationship  existed  between 
who  answered  the  questions  and  the  urgency  of  the  patient 
(hospital  A,  chi  square  =  0.02,  df  =  1,  p  >  0.05;  hospital 
B,  chi  square  =  0.00,  df  =  1,  p  >  0.05). 

However,  the  question  of  who  responded  -  the  patient 
or  his  companion  -  was  significantly  dependent  upon  the  age 
of  the  patient.  Pediatric  patients  (less  than  16  years) 
were  more  likely  to  have  their  companion  answer  the  questions 
(chi  square  =  233.33,  df  =  1,  p  <  0.05).  However,  when  "who 
answered"  was  compared  to  the  urgency  of  the  patient, 
controlling  for  age,  still  no  significant  differences  were 
found  at  the  0.05  level  (adult,  chi  square  =  2.32,  df  =  1; 
child,  chi  square  =  2.29,  df  =  1).  Caution  must  be  exercised 
in  attempting  to  draw  conclusions  from  this  information  since 
observations  were  missing  for  65  (19%)  of  the  336  adult  cases. 

Length  of  Time  Aware  of  Problem 

A  significant  relationship  was  observed  between  the 
patient's  "true  urgency"  and  the  length  of  time  the  patient 
reported  that  he  had  been  aware  of  his  medical  problem. 

Table  9  reports  these  findings.  This  was  not  unexpected 
since  one  of  the  criteria  for  nonurgency  was  that  the  patient 
had  his  condition  for  24  hours  or  more  (Appendix  A) .  Due  to 
this  fact,  correlations  larger  than  those  found  (hospital  A, 
Kendall's  tau  =  0.353?  hospital  B,  tau  =  0.321)  would  not 
have  been  unreasonable. 
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LENGTH  OF  TIME  PATIENT  HAS  BEEN  AWARE  OF  PROBLEM  BY  TRUE 

URGENCY  RATING 


Emergent  Urgent  Nonurgent  Totals 


Hospital  A 


<  3  hours 

3 

(  2.2%) 
(75.0%) 

93 

(68.4%) 

(69.9%) 

40 

(29.4%) 

(35.7%) 

136 

6-24  hours 

0 

(  0.0%) 

(  0.0%) 

21 

(56.8%) 

(15.8%) 

16 

(43.2%) 

(14.3%) 

37 

1-7  days 

1 

(  2.0%) 
(25.0%) 

13 

(26.0%) 
(  9.8%) 

36 

(72.0%) 

(32.1%) 

50 

>  week 

0 

(  0.0%) 

(  0.0%) 

6 

(23.1%) 
(  4.5%) 

20 

(76.9%) 

(17.9%) 

26 

Totals 

4 

133 

112 

249 

Hospital  B 

<  3  hours 

3 

(  1.9%) 
(75.0%) 

102 

(63.0%) 

(76.1%) 

57 

(35.2%) 

(44.9%) 

162 

6-24  hours 

1 

(  2.3%) 
(25.0%) 

19 

(43.2%) 

(14.2%) 

24 

(54.5%) 

(18.9%) 

44 

1-7  days 

0 

(  0.0%) 

(  0.0%) 

10 

(21.7%) 
(  7.5%) 

36 

(78.3%) 

(28.3%) 

46 

>  week 

0 

(  0.0%) 

(  0.0%) 

3 

(23.1%) 
(  2.2%) 

10 

(76.9%) 

(  7.9%) 

13 

Totals 

4 

134 

127 

265 

Note.-  Chi 

square  is  38, 

8,  df  =  3, 

P  < 

0.001  when 

the 

emergent  and  urgent  categories  are  combined. 


Note.-  Row  percentages  are  given  to  the  right  of  the  cell 
sizes.  Beneath  the  row  percentages  are  column  percentages. 
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Medical  Facility  Required:  Hospital  A 


In  hospital  A  only,  patients  were  asked  whether 
they  thought  they  might  have  to  stay  in  hospital  (cf.  p.37). 
If  they  responded  negatively,  they  were  asked  whether  an 
appointment  with  a  doctor  of  their  choosing  within  the  next 
three  days  would  have  been  as  convenient.  Table  10 
summarizes  the  relationship  between  their  responses  and 
"true  urgency".  A  relationship  existed  (Kendall's  tau  = 

0 . 366 ,p<0 . 05)  between  the  patients'  answers  to  these 
questions  and  their  "true  urgency".  The  corresponding 
percentage  of  agreement  between  the  patient  responses  and 
the  "true  urgency"  ratings  was  61%,  (i.e.  sum  of  diagonal 

percentages  in  Table  10) ,  which  is  within  the  range  of  the 
staff  percentages  of  agreement  with  the  "true  urgency" 
ratings  in  hospital  A.  Therefore,  these  questions  might  be 
valuable  as  one  component  in  a  triage  system. 

Composite  Patient  Urgency  Rating:  Hospital  A 

In  hospital  A,  both  the  length  of  time  the  patient 
had  been  aware  of  his  problem  and  the  type  of  facility  he 
believed  he  required  correlated  with  the  "true  urgency" 
ratings  better  than  they  correlated  with  each  other  (Table 
11).  Therefore,  these  two  variables  ("time  aware"  and 
"facility  required")  were  combined  to  form  a  "composite 
patient  urgency  rating"  (cf.  p.  26).  These  criteria  class 
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TABLE  10 

HOSPITAL  A:  PATIENT’S  PERCEPTION  OF  MEDICAL  FACILITY  REQUIRED 

BY  TRUE  URGENCY  RATING 


Pt.  thinks 
he  may  have 
to  stay  in 
hospital 


Dr.'s  appt . 
in  next  3 
days  is  not 
convenient 


Dr.'s  appt. 
in  next  3 
days  is  as 
convenient 


Totals 


Emergent 

4 

( 

1.  6%) 

1 

(  0.4%) 

0 

(  0.0%) 

5 

(  2.0%) 

Urgent 

20 

( 

8.0%) 

98 

(39. 4%) 

15 

(  6.0%) 

133 

(53. 4%) 

Nonurgent 

6 

( 

2.  4%) 

56 

(22.  5%) 

49 

(19.7%) 

111 

(44.6%) 

Totals 

30 

(12. 0%) 

155 

(62.  3%) 

64 

(25.  7%) 

249 

(100.0%) 

Note.-  The  total  percentage  of  agreement  between  these 
patient  assessments  and  the  "true  urgency"  ratings  is  61% 
(i.e.,  sum  of  diagonal  percentages).  Kendall's  tau  between 
these  ratings  is  0.366. 
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TABLE  11 


HOSPITAL  A: 

AGREEMENT 

AND 

BETWEEN  THE 
TRUE  URGENCY 

PATIENT  URGENCY 
RATING 

VARIABLES 

Length  of 
time  with 
problem 

Facility 

required 

Composite 

patient 

urgency 

True 

urgency 

Length  of 
time  with 
problem 

1.000 

*** 

*** 

*  ** 

Facility 

required 

0.236 
(N  =  272) 

1.000 

87% 

62% 

Composite 

patient 

urgency 

0.503 
(N  =  270) 

0.812 
(N  =  266) 

1.000 

68% 

True 

urgency 

0.353 
(N  =  249) 

0.366 
(N  =  249) 

0.473 
(N  =  247) 

1.000 

***  Percentages  of  agreement  could  not  be  obtained 
because  one  variable  had  three  values  and  the  other  had 
four  values. 
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the  patient  as  "emergent"  if  he  has  had  his  problem  for  less 
than  24  hours  and  thinks  he  may  have  to  stay  in  hospital, 
and  as  "nonurgent"  if  an  appointment  in  a  doctor's  office 
would  have  been  as  convenient  for  him  or  if  such  an 
appointment  was  not  as  convenient  but  he  had  his  problem  for 
more  than  24  hours.  Other  possible  responses  were  coded  as 
"urgent" . 

The  correlation  between  this  indirect  patient 
urgency  rating  and  the  "true  urgency"  rating  was  0.473,  with 
a  corresponding  percentage  of  agreement  of  68%.  This 
percentage  is  within  the  95%  confidence  bounds  of  the 
percentages  of  agreement  between  the  staff  and  "true  urgency" 
ratings  for  hospital  A  (58.1%-  76.6%).  Thus,  it  appears 
that  patient  self  classification,  employing  predetermined 
criteria,  can  be  as  accurate  as  staff  classifications  of 
urgency.  Before  self  triage  is  taken  more  seriously  though, 
two  questions  must  be  answered:  (1)  what  is  the  largest 
tolerable  amount  of  error  acceptable  in  any  triage  system, 
and  (2)  will  the  implementation  of  a  triage  system  alter  the 
accuracy  of  the  staff's  and/or  patient's  perceptions?  Only 
evaluation  after  implementation  can  answer  the  latter 
question. 

A  note  of  caution  must  be  included  at  this  point 
concerning  the  extent  of  the  accuracy  of  the  patient 
responses.  Nurses  asked  the  patients  the  self  triage 
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questions  and  recorded  the  patient  responses.  The  nurses 
were  not  trained  interviewers  and  thus  the  responses  may 
not  have  been  entirely  objective.  Since  the  agreement 
between  patient  and  nurse  (Kendall  correlation  coefficient 
is  0.538)  is  higher  than  between  the  patient  and  the  "true 
urgency"  rating  (0.473)  and  between  the  nurse  and  "true 
urgency"  rating  (0.463),  such  a  source  of  invalidity  should 
not  be  overlooked. 

Physician  Contacted  Prior  to  Visit:  Hospital  B 

In  hospital  B,  the  patient  response  to  the  questions 
relating  to  the  facilities  he  felt  he  required  were  replaced 
by  questions  on  the  patient's  attempt  to  contact  a  doctor 
before  coming  to  the  emergency  department  (cf.  p.  37). 

Eleven  percent  of  the  patients  reported  contacting  a  doctor 
but  no  significant  relationship  was  found  between  the 
patient's  urgency  and  whether  he  contacted  a  doctor  prior  to 
visiting  the  emergency  department  (Table  12,  chi  square  = 
0.00,  probability  >0.90).  Other  studies  (Bain  &  Johnson, 
1971,  p.  36;  Weinerman,  Ratner,  Robbins,  &  Lavenhar,  1966, 
p.  1050;  Vayda,  Gent,  &  Paisley,  1972,  p.  13)  have  reported 
that  physicians  appear  to  act  as  a  screen  by  discouraging 
patients  with  nonurgent  conditions  from  visiting  the 
emergency  department.  This  does  not  appear  to  be  the  case 
for  hospital  B. 
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TABLE  12 


HOSPITAL  B:  PATIENT'S  ATTEMPT  TO  CONTACT  A  DOCTOR  BEFORE 

COMING  TO  THE  EMERGENCY  DEPARTMENT  BY  TRUE 
URGENCY  RATING 


Tried  to  contact 
doctor 


Did  not  try  to 
contact  doctor 


Emergent 

1 

(  3.6%) 

(25.0%) 

(  0.4%) 

3 

(  1.3%) 
(75.0%) 
1.2%) 

Urgent 

13 

(46.4%) 

(  9.8%) 

(  5.0%) 

119 

(51.1%) 

(90.2%) 

(45.6%) 

Nonurgent 

14 

(50.0%) 

(11.2%) 

(  5.4%) 

111 

(47.6%) 

(88.8%) 

(42.5%) 

Total 

28 

233 

Note.-  Percentages  are  by  column,  row  and  total 
respectively. 

Note.-  Test  of  significance  between  proportion 
(Ferguson,  1971,  p.  160-162)  of  nonurgent  patients  in  each 
group  was  not  significant.  z  is  0.49  (p  >  0.05). 
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Summary  of  Patient’s  Perception  of  Urgency 

If  patient  self  triage  is  considered,  guidelines  for 
the  patient's  decision  must  be  formulated.  Two  questions 
which,  in  combination  with  others,  might  be  useful  as 
screening  questions  are:  "How  long  have  you  been  aware  of 
the  medical  problem  which  brought  you  to  the  emergency 
department  today?"  and  "Do  you  think  that  you  might  have  to 
stay  in  hospital?  If  not,  would  an  appointment  in  a  doctor's 
office  be  as  convenient  for  you?"  Questions  of  prior 
physician  contact  and  the  need  for  the  person  accompanying 
the  patient  to  answer  for  the  patient  do  not  appear  to  be 
useful  criteria. 

Factors  Influencing  the  Accuracy  of  the  Urgency  Ratings _ 

To  repeat,  caution  must  be  exercised  in  interpreting 
the  preceding  reported  measures  of  agreement  since:  (1)  the 
error  component  of  the  "true  urgency"  ratings  is  not 
definitive,  and  (2)  the  ability  of  the  staff  to  assess 
urgency  may  be  dependent  on  factors  which  have  not  been 
controlled  (e.g#/the  type  of  patients  being  assigned  ratings). 
The  final  section  of  this  chapter  discusses  a  few  examples  of 
factors  which  may  influence  the  accuracy  of  the  urgency  ratings. 

An  error  component  in  the  "true  urgency"  ratings  could 
arise  from  inadequate  patient  record  information.  In  hospital 
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A,  the  panel  physicians  noted  when  the  patient  record 
information  was  inadequate  and  did  not  allow  them  to  be 
confident  in  their  assessments  (cf .  p.  36) .  Out  of  the  250 
records  assigned  "true  urgency"  ratings,  there  were  81 
records  for  which  one  panel  physician  indicated  the 
information  was  inadequate,  three  records  for  which  two 
panel  physicians  stated  the  information  was  inadequate  and 
no  record  for  which  the  three  panel  physicians  reported 
the  information  was  inadequate.  The  patients  were 
dichotomized  into  those  for  whom  at  least  one  panel 
physician  believed  the  information  on  their  patient  record 
was  inadequate  and  those  who  received  no  such  demarcation.. 

The  percentages  of  agreement  between  the  "true  urgency" 
ratings  and  the  s.taff  ratings  in  hospital  A  were  then 
calculated  for  these  two  groups  of  patients  (Table  13). 

Using  the  proportions  test,  a  significant  difference  in  the 
percentages  of  agreement  was  not  found  for  any  of  the  staff 
members  (clerk,  z  =  0.02,  p  =  0.98;  nurse,  z  =  0.14,  p  =  0.88? 
doctor,  z  =  0.92,  p  =  0.35).  Thus,  there  is  less  reason  to 
believe  that  the  completeness  of  patient  records  influenced 
the  accuracy  of  the  "true  urgency"  ratings  in  hospital  A. 

Other  sources  of  error  may  have  been  present  in  the 
"true  urgency"  ratings  which  would  have  influenced  the 
reported  measures  of  staff  accuracy. 
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TABLE  13 


PERCENTAGES  OF  AGREEMENT  BETWEEN  STAFF  URGENCY  RATINGS  AND 
TRUE  URGENCY  RATINGS  DICHOTOMIZED  BY  ADEQUACY  OF  PATIENT 
RECORD  INFORMATION  FOR  HOSPITAL  A 


Clerk  Nurse  Physician 


Patient  record  information 
inadequate  for  at  least 
one  panel  physician  to 
be  confident  in  his  rating 


61.9%  69.0%  66.7% 

(n=83)  (n=84)  (n=84) 


Patient  record  information 
adequate  for  all  panel 
physicians 


62.0%  69.9% 

(n=166)  (n=166) 


72.3% 

(n=166) 
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The  patients  assigned  ratings  in  this  study  were  not 
randomly  selected.  Consequently,  the  validity  of  the 
measures  of  agreement  for  the  staff  is  dependent  on  the 
validity  of  the  assumption  that  patient  mix  does  not 
influence  the  ability  of  the  staff  to  rate  the  urgency  of 
patients . 

One  finding  which  indicated  that  patient  mix  may 
influence  the  ability  of  staff  to  assess  urgency  was  the 
low  Kendall  correlations  between  the  staff  ratings  and  the 
"true  urgency"  ratings  on  the  night  shift  (Table  14) .  Three 
of  the  fifteen  Kendall  correlations  between  staff  and  "true 
urgency"  ratings  by  shift  had  probabilities  greater  than 
0.001.  These  three  were  all  obtained  on  the  night  shift 
(nurse,  hospital  A,  tau  =  0.190,  p  =  0.063;  doctor,  hospital 
A,  tau  =  0.221,  p  =  0.024;  doctor,  hospital  B,  tau  =  0.266, 
p  =  0.009).  If  persons  entering  the  emergency  department 
at  night  differed  from  those  arriving  at  other  times,  this 
difference  in  patient  mix  may  have  decreased  the  accuracy  of 
the  staff  in  assigning  urgency  ratings. 

It  must  be  noted  that  another  hypothesis  for  the  low 
correlations  on  the  night  shift  is  that  they  arise  from 
variations  in  the  ability  of  individual  staff  members.  This 
investigator  believes  both  tentative  explanations  are  worth 
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TABLE  14 

KENDALL  CORRELATION  COEFFICIENTS  BETWEEN  THE  STAFF  AND  TRUE 
URGENCY  RATINGS  BY  SHIFT* 


Day 

Evening 

Night 

Clerk 

-  hospital 

A 

0.427 
(N=85 ) 

0.397 

(N=132) 

0.409 
(N=32 ) 

Nurse 
-  hospital 

A 

0.491 

(N=86) 

0.493 
(N=132 ) 

0.190** 
(N=32 ) 

-  hospital 

B 

0.490 

(N=97) 

0.304 
(N=129 ) 

0.  221 
(N=39) 

Doctor 
-  hospital 

A 

0.684 
(N=97 ) 

0.368 

(N=121) 

0.442 

(N=32) 

-  hospital 

B 

0.467 

(N=97) 

0.486 
(N=129 ) 

0.266 
(N=39 ) 

*  Hours  included  in  Day,  Evening  and  Night  Shifts 


hospital  A 
(clerk  &  nurse) 

hospital  A 
(doctor) 

hospital  B 

Day 

7.01  -  15.00 

8.01  -  16.00 

8.01  -  16.00 

Evening 

15.01  -  23.00 

16.01  -  24.00 

16.01  -  24.00 

Night 

23.01  -  7.00 

0.01  -  8.00 

0.01  -  8.00 

**  This  is  the  only  correlation  which  is  not  significant  at 
the  0.05  alpha  level. 
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investigating . 


In  hospital  A,  it  appeared  that  male  patients  were 
easier  for  the  staff  to  classify.  This  was  particularly 
true  for  the  clerk  (males,  tau  =  0.5161;  females,  tau  = 
0.2465)  and  at  least  obvious  for  the  doctor  (males,  tau  = 
0.5040;  females,  tau  =  0.4690).  In  hospital  B,  the 
correlations  did  not  differ  between  the  nurse  and  doctor  by 
more  than  0.08.  Therefore,  it  appears  that  the  sex  of  the 
patient  may  be  related  to  the  accuracy  of  the  urgency 
ratings  for  the  clerk  and  possibly  . the  nurse  in  hospital  A, 
but  had  little  effect  on  the  accuracy  of  the  ratings  of  the 
other  staff  members. 

While  this  study  was  not  designed  to  provide  a 
detailed  analysis  of  the  validity  of  the  urgency  measures, 
these  examples  provided  a  report  of  some  areas  of  concern. 

Summary 

The  analyses  for  this  chapter  were  based  on  249 
cases  for  the  clerk,  250  cases  for  the  nurse  and  doctor  in 
hospital  A,  and  265  cases  for  the  nurse  and  doctor  in  . 
hospital  B.  In  each  hospital,  out  of  all  patients  given  an 
urgency  rating  by  the  nurse  and  doctor,  275  were  randomly 
selected  for  review  by  the  panel.  Some  of  the  charts  for 
these  patients  could  not  be  retrieved  because  patient  names 
or  numbers  were  illegible,  or  charts  were  missing  due  to 
misfiling  or  subsequent  use.  One  case  in  each  hospital  was 
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omitted  because  all  the  panel  physicians  differed  in  their 
ratings.  Another  case  in  hospital  A  was  omitted  because  one 
of  the  three  panel  physicians  did  not  provide  a  rating 
because  he  believed  the  information  on  the  patient's  chart 
was  inadequate. 

Unanimity  amongst  the  panel  physicians  was  lacking 
in  over  40%  of  the  cases.  Only  one  patient  in  each  hospital 
was  assigned  an  "emergent1'  rating  by  all  three  panel  members. 
Slightly  more  than  a  quarter  of  the  patients  were  labelled 
"urgent",  and  approximately  another  quarter  were  classed  as 
"nonurgent"  by  all  three  panel  physicians. 

A  statistically  significant  relationship  existed 
between  the  urgency  ratings  of  the  three  types  of  staff  and 
the  "true  urgency"  ratings  derived  from  the  panel 
assessments.  Comparisons  between  the  measures  of  accuracy 
for  the  nurse  and  physician  showed  no  significant  differences. 
The  accuracy  of  the  clerk  approached  that  of  the  nurse  and 
physician  although  she  tended  to  be  conservative  in  her 
ratings.  She  had  a  lower  level  of  accuracy  which  was 
statistically  significant  for  three  of  four  comparisons  with 
health  professional  percentages  of  agreement. 

Experience,  as  defined  in  this  study,  did  not 
influence  the  ability  of  the  staff  to  rate  urgency. 

For  hospital  A  a  composite  variable  was  derived  from 
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responses  by  the  patients  on  the  perception  of  the  urgency 
of  their  conditions.  This  patient's  perception  of  urgency 
variable  had  a  Kendall  correlation  (0.443)  and  percentage 
of  agreement  (68%)  with  the  "true  urgency"  ratings  which 
were  within  the  range  of  values  for  the  staff  (tau:  0.434  - 
0.494;  percentages  of  agreement:  62%  -  70%).  Following 
this,  a  brief  discussion  of  factors  which  may  have 
influenced  the  accuracy  of  the  urgency  ratings  was  given. 

The  next  chapter  summarizes  the  objective,  methods 
and  findings  of  this  study,  and  provides  recommendations 
for  further  investigation  in  the  area  of  classifying 
emergency  department  patients. 
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CHAPTER  FIVE 


SUMMARY  AND  RECOMMENDATIONS 
Summary 

The  number  of  patients  seeking  care  in  emergency 
departments  has  been  increasing.  In  an  attempt  to  improve 
the  efficiency  of  emergency  departments,  triage  officers 
have  been  advocated  in  some  hospitals  to  direct  patients 
into  specialized  treatment  areas.  Very  little  has  been 
written  on  the  degree  of  training  and  experience  required 
to  adequately  triage  patients.  Training  and  experience  of 
triage  officers  was  not  examined  in  this  study;  rather  the 
investigator  examined  the  accuracy  of  emergency  department 
staff  presently  working  in  emergency  departments  in  sorting 
patients  by  urgency. 

The  staff  of  two  Edmonton  emergency  departments 
agreed  to  co-operate  in  this  study.  In  one  hospital 
(hospital  A)  the  emergency  physicians,  nurses,  and  the 
registration  clerks  participated,  while  in  the  second 
hospital  (hospital  B)  only  the  emergency  physicians  and 
nurses  were  involved.  The  patient  sample  consisted  of  all 
persons  seen  by  emergency  physicians  during  a  predetermined 
time  period  for  which  it  was  estimated  that  350  patients 
would  be  seen.  These  patients  were  not  randomly  selected 
but  taken  consecutively  as  they  entered  the  emergency 
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department. 

The  time  period  for  the  data  collection  was  three 
days  for  the  first  hospital  and  seven  days  for  the  second. 

Both  studies  were  carried  out  in  two  different  time  periods 
of  June  1973. 

Criteria  for  the  urgency  ratings  employed  (emergent, 
urgent,  nonurgent)  were  developed  by  Vayda,  Gent  and  Paisley 
(1972) . 

During  the  data  collection  phase,  the  clerk  who 
registered  a  study  patient  in  hospital  A  indicated  whether 
she  believed  the  patient  was  emergent,  urgent,  or  nonurgent 
on  an  assessment  form  (Appendix  B-l).  She  also  noted  the 
length  of  time  she  had  worked  in  an  emergency  department  by 
checking  one  of  four  length  of  experience  categories:  less 
than  three  months,  three  months  to  one  year,  one  or  two  years, 
or  more  than  two  years.  The  clerk  also  wrote  the  patient 
registration  number  on  the  top  of  her  assessment  form  and  on 
top  of  blank  nurse  and  physician  assessment  forms.  She  then 
placed  her  assessment  form  in  a  box  at  the  registration  desk 
and  attached  the  nurse  and  physician  forms  to  the  patient 
record.  In  both  hospitals,  the  nurse  that  accompanied  the 
patient  to  an  examination  cubicle  indicated  whether  she 
believed  this  patient  was  emergent,  urgent,  or  nonurgent  on 
the  nurse  assessment  form  (Appendix  B-2  and  B-3)  attached  to 
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the  patient's  record.  The  nurse  also  indicated  how  long 
she  had  worked  in  an  emergency  department  by  checking  one 
of  the  same  four  experience  categories  used  on  the  clerk's 
assessment  form.  The  nurse  was  also  responsible  for  asking 
the  patient  questions  which  were  used  to  provide  an  indirect 
measure  of  the  patient's  perception  of  the  urgency  of  his 
condition.  These  questions  were  not  identical  in  the  two 
hospitals.  After  the  nurse  had  completed  her  form,  she 
detached  it  from  the  patient  record  and  placed  it  in  a  box 
provided  at  the  nursing  station. 

The  emergency  physician  who  examined  the  patient 
rated  the  patient's  urgency,  noted  how  long  he  had  worked  in 
an  emergency  department,  detached  this  assessment  form  from 
the  patient's  record,  and  placed  it  in  a  box  provided  at  the 
physician's  desk.  Emergency  physicians  (for  the  purpose  of 
this  study)  were  restricted  to  full  or  part  time  salaried 
physicians  who  worked  in  the  emergency  department.  It  did 
not  include  physicians  who  saw  their  own  patients  in  the 
emergency  department. 

To  estimate  the  accuracy  of  the  staff  members  in 
classifying  the  urgency  of  emergency  department  patients,  a 
"true  urgency"  assessment  was  derived.  To  obtain  an 
estimate  of  "true  urgency",  the  records  of  275  patients  which 
had  been  assessed  by  all  participating  staff  members,  were 
randomly  selected.  These  records  were  independently  reviewed 
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by  a  panel  of  three  physicians  in  each  hospital.  The 
physicians  rated  each  patient  as  emergent,  urgent,  or 
nonurgent  utilizing  the  same  urgency  criteria  as  used  by  the 
staff.  The  "true  urgency"  rating  for  this  study  was  then 
defined  to  be  the  rating  assigned  by  at  least  two  of  these 
three  panel  physicians.  Of  the  275  records  selected  for 
review  in  each  hospital,  24  records  could  not  be  located  in 
hospital  A  and  9  in  hospital  B. 

One  patient  in  each  hospital  was  omitted  from  further 
analysis  because  all  three  panel  physicians  differed  on  their 
assessment  of  the  urgency  of  the  patient.  In  addition,  one 
patient  in  hospital  A  was  not  assigned  a  "true  urgency"  rating 
by  one  of  the  panel  physicians  because  he  believed  the 
information  on  the  patient's  record  was  not  sufficient  for 
such  an  assessment.  Consequently,  the  analysis  of  the  study 
was  based  on  the  assessments  of  250  patients  in  hospital  A 
and  265  patients  in  hospital  B. 

When  the  panel  physician  ratings  were  examined,  it 
was  found  that  only  one  patient  in  each  hospital  was  assigned 
an  "emergent"  rating  by  all  three  physicians.  About  30%  of 
the  patients  were  classified  as  "urgent"  by  all  panel  members. 
Approximately  a  quarter  were  labelled  "nonurgent"  by  all  three 
panel  physicians.  The  panel  physicians  did  not  agree 
unanimously  in  over  40%  of  the  ratings. 


Three  measures  of  agreement  were  used  to  report  the 
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ability  of  the  staff  to  assess  urgency.  They  were  the 
Kendall's  correlation  coefficient,  percentages  of  agreement 
and  the  chi  square  goodness  of  fit  test.  The  Kendall 
correlation  coefficients  between  the  staff  and  the  "true 
urgency"  ratings  in  hospital  A  ranged  from  0.350  for  the 
clerk  to  0.494  for  the  physician.  The  percentages  of 
agreement  between  the  staff  and  "true  urgency"  ratings 
ranged  from  62%  to  70%.  Using  a  two-tailed  proportions 
test  no  significant  difference  was  observed  between  the 
nurses  and  doctors.  The  ability  of  the  clerk  approached 
that  of  the  health  professionals,  although  she  was  more 
conservative  in  her  assessments  of  urgency. 

The  chi  square  goodness  of  fit  test  was  used  to 
determine  which  staff  members  were  most  accurate  in 
estimating  the  proportion  of  persons  (but  not  necessarily 
the  same  persons)  falling  into  the  nonurgent,  urgent,  and 
emergent  categories.  Using  a  level  of  significance  of  0.05 
one  would  conclude  that  the  clerk,  the  emergency  physician 
in  both  hospitals,  and  the  three  of  the  panel  physicians 
were  not  able  to  accurately  estimate  the  "true"  proportion 
of  patients  in  each  urgency  category.  It  should  be  noted 
here  that  the  panel  physicians  categorized  the  patients 
post  hoc  from  the  patient  records.  Therefore,  the  informa¬ 
tion  available  to  them  differed  from  that  available  to  the 
staff  who  classified  the  patients  while  they  were  in  the 
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emergency  department.  As  a  result  of  these  findings,  one 
might  question  the  proportion  of  emergent,  urgent,  and 
nonurgent  patients  that  have  been  reported  in  other  studies. 

The  clerk's  categorizations  were  examined  further  and 
it  was  found  that  she  had  a  tendency  to  place  the  patients 
in  categories  of  greater  urgency  than  the  panel.  The  clerk 
was  the  only  staff  member  in  either  hospital  to  label  all 
four  patients  "emergent"  whose  "true  urgency"  was  defined 
to  be  emergent. 

The  second  major  component  of  this  study  was  to 
investigate  the  effect  of  experience  on  the  ability  of  staff 
to  rate  urgency.  From  the  data  collected,  experience  did 
not  appear  to  account  for  differences  in  the  ability  to 
assess  urgency. 

As  noted  earlier,  the  nurses  asked  the  patients 
questions  related  to  the  patient's  perception  of  his  own 
urgency.  In  hospital  A  the  patients  were  asked  (1)  the 
length  of  time  they  had  been  aware  of  the  problem  which 
brought  them  to  the  emergency  department,  (2)  whether  or 
not  they  thought  they  would  have  to  be  admitted  to  hospital, 
and  (3)  if  they  didn't  think  they  would  have  to  be 
hospitalized,  whether  an  appointment  in  a  doctor's  office 
in  the  next  three  days  would  have  been  as  convenient.  These 
variables  were  combined  as  follows  to  form  a  composite 
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patient  urgency  rating.  (1)  If  the  patient  stated  that  he 
had  had  his  problem  for  less  than  24  hours  and  thought  that 
he  might  have  to  stay  in  hospital,  he  was  given  an  "emergent" 
rating.  (2)  If  he  thought  that  an  appointment  in  a  doctor's 
office  would  have  been  as  convenient  for  him,  or  if  such  an 
appointment  would  not  have  been  as  convenient  but  he  had 
his  problem  for  more  than  24  hours,  he  was  classified  as 
"nonurgent".  (3)  All  other  possible  response  combinations 
were  coded  as  "urgent". 

This  composite  patient  urgency  rating  correlated 
0.473  with  the  "true  urgency"  ratings.  The  corresponding 
percentage  of  agreement  with  the  "true  urgency"  ratings  was 
68%  which  was  within  the  percentages  of  agreement  for  the 
staff  of  hospital  A.  It  appeared  that  patient  self 
classification,  employing  predetermined  criteria,  was  as 
accurate  as  the  staff  members'  classification  of  urgency. 

In  hospital  B,  the  patients  were  asked  whether  they 
had  contacted  a  doctor  prior  to  coming  to  the  emergency 
department.  Other  studies  have  reported  that  physicians 
appear  to  act  as  a  screen  by  discouraging  patients  with 
nonurgent  conditions  from  visiting  an  emergency  department. 
This  was  not  the  case  in  hospital  B.  Here,  no  significant 
difference  was  found  between  the  proportion  of  nonurgent 
patients  who  tried  to  contact  a  doctor  and  the  proportion 
of  patients  who  did  not  try  to  contact  their  doctor. 
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This  study  did  not  attain  the  external  validity  required 
(Bracht  &  Glass,  1968)  to  generalize  results  to  other  emergency 
departments . 

Recommendations 

This  study  has  demonstrated  the  need  for  further 
research  in  the  following  areas. 

(1)  Valid  "true  urgency"  ratings  must  be  developed,  since  any 
measures  of  accuracy  in  estimating  "true  urgency'1'  are  dependent 
upon  the  validity  of  the  "true  urgency"  ratings.  One  could 
improve  the  "true  urgency"  classification  by  reviewing  cases  of 
evaluator  discrepancy  and  thereby  refining  the  criteria  for 
"emergent",  "urgent"  and  "nonurgent". 

(2)  Consideration  should  be  given  to  determining  the  validity 

of  a  record  review  as  compared  to  direct  observation.  This  could 
be  accomplished  by  having  panel  physicians  observe  a  sample  of 
patients  as  they  enter  the  emergency  department  and  rate  them 
in  a  manner  similar  to  the  way  the  emergency  physicians  did  in 
this  study.  After  time  had  elapsed,  so  that  the  physicians  would 
have  difficulty  recalling  the  conditions  of  most  of  the  patients 
and  so  that  further  information  (e.g.,  laboratory  and  x-ray 
results,  return  visits,  admission  to  hospital)  would  be  noted 
on  the  patient  records,  the  panel  would  review  the  records  of 
these  patients  and  again  assign  urgency^ ratings. ,  Thus,  data 
would  be  provided  for  comparing  assessments  made  from  a  record 
review  to  those  made  by  direct  observation. 

(3)  To  improve  the  accuracy  of  classifying  the  urgency  of 
patients,  factors  that  influence  the  ability  of  staff  and 
patients  in  assessing  urgency  must  be  determined.  For  example, 
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patient  mix  might  be  investigated  with  the  objective  of 
developing  easily  applied  patient  mix  criteria  which  would 
identify  patients  who  are  difficult  to  classify.  Triage  could 
then  be  restricted  to  patients  for  whom  the  accuracy  of  the 
classification  was  considered  adequate. 

(4)  Further  attention  should  be  given  to  examining  the  accuracy 
and  feasibility  of  patients  assessing  the  urgency  of  their  own 
medical  conditions,  especially  now  when  increased  consumer 
participation  is  being  advocated. 

(5)  Finally,  the  study  pointed  out  that  the  urgency  criteria 
employed  (cf.  Appendix  A)  did  not  separate  patients  into  three 
proportionate  groups.  If  benefits  are  to  be  gained  by  a  triage 
system,  means  of  separating  patients  into  groups  of  sufficient 
size  to  warrant  specialized  treatment  units  must  be  found. 

The  patient  demands  being  placed  on  emergency 
departments  has  accentuated  the  need  to  sort  patients 
according  to  the  urgency  of  their  conditions.  This  study  has 
shown  that  the  patients  themselves,  the  emergency  department 
nurses,  the  emergency  physicians,  and  possibly  the  registration 
clerks,  were  equally  capable  in  classifying  the  urgency  of 
patient  conditions.  Length  of  experience  in  an  emergency  did 
not  influence  the  classification  ability  of  the  staff.  These 
results  cannot  be  generalized  to  other  hospitals  but  this 
study  has  provided  a  basic  design  for  further  investigation 
into  factors  influencing  the  ability  to  classify  the  urgency 
of  emergency  department  patients. 
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APPENDIX  A 


■ 


i,l 


I  • 


■  -7 


A: 


Definitions  and  examples  of  emergent ,  urgent,  and 

nonurgent  conditions* 


EMERGENT  Condition  requires  immediate  medical  attention; 
time  delay  is  harmful  to  patient; 
disorder  is  acute  and  potentially  threatening  to 
life  or  function; 

cases  of  this  type  will  usually  require 
hospitalization. 

Examples:  Unconsciousness 

Rapid  change  in  level  of  consciousness 
Severe  crushing  chest  pain 
Arrythmia  or  tachycardia 
Shock 

Pulmonary  edema 
Poisoning 
Perforated  viscus 

Gunshot  wound  involving  a  major  organ 
Major  accidental  or  traumatic  injury 
Head  injury  with  unconsciousness  or  rapid  change 
in  level  of  consciousness 
Major  hemorrhage  with  or  without  trauma, 
regardless  of  site 

Acute  arterial  embolus  or  thrombosis 
Acute  burn  -  second  or  third  degree  -  10%  or 
more  of  body  surface 
Suicide  attempt 
Foreign  body  in  trachea 
Convulsions 


URGENT  Condition  requires  medical  attention  within  a 

few  hours 

there  is  possible  danger  to  patient  or  ultimate 
outcome  is  not  promptly  medically  attended; 
disorder  is  of  acute  onset  but  not  necessarily 
severe  or  life  threatening; 
patients  will  not  usually  but  may  require 
hospitalization. 

Examples :  Chest  pain  or  recent  onset  or  of  suddenly 

increasing  severity 
Shortness  of  breath  of  recent  onset 
Semi-consciousness  or  stupor 
Laceration  of  less  than  24  hours  requiring 
sutures  or  dressing 

Injury  less  than  24  hours  with  severe  pain, 

severe  limitation  of  motion,  swelling  and 
the  possibility  of  a  fracture  requiring  x- 
ray 
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Fracture  of  recent  onset  requiring  treatment 
Acute  abdominal  pain  of  recent  onset  -  or  acute 
abdomen 

Foreign  body  in  eye  of  less  than  24  hours 
duration 

Auto  accident  of  recent  onset  (24  hours) 
requiring  evaluation 
Depression  with  threat  of  suicide 
Acute  psychosis 

Neurosis  not  manageable  by  family  or  physician 
High  fever  of  sudden  onset  (>103°F) 

Persistent  diarrhea  or  vomiting  -  severe  and  of 
sudden  onset 

Rape 

Incomplete  abortion  without  major  hemorrhage 


NONURGENT  Condition  does  not  require  the  resources  of  an 

emergency  service; 

symptoms  of  long  duration  without  sudden  change 
in  severity 

referral  for  routine  medical  care  is  all  that  is 
needed; 

disorder  is  not  acute,  it  is  minor  in  severity; 

routine  care  in  a  physician's  office  or  no 
medical  care  is  required. 

Examples :  Upper  respiratory  infection 

Sore  foot  for  5  days  (regardless  of  cause) 

Prescription  renewal 

Fever,  mild  or  immediate  in  nature  (99 °F  - 
102. 9°F) 

Sprains,  minor  bruises  or  small  cuts  not 
requiring  x-ray  or  suturing 

Minor  injuries  or  cuts  of  greater  than  24  hour 
duration 

Ill  defined  symptoms  especially  if  over  24  hours 
in  duration 


Source :  Vayda,  E.,  Gent,  M.  &  Paisley,  L.  St.  Joseph's 

Hospital  -  McMaster  emergency  study:  I  Methodology 

and  findings.  Unpublished  manuscript,  McMaster 

University,  Department  of  Clinical  Epidemiology  and 
Biostatistics,  1972. 
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n-]  : 


Clerk's  Urgency  Rating  Form 


Chart  Number  _ 

CLFRK'S  RATING  FORM 

Usina  the  predetermined  classification, 
please  indicate  the  category  to  which, 
you  believe,  this  patient  belongs. 

[|  EMERGEN^  [j  URGENT  □NONURGENT 

How  lonq  have  you  worked  in  an  emergency 
department? 

□  less  than  3  months  Dl  to  2  years 

[_J  3  months  to  1  year  [Hmore  than  ?  years 


When  used,  this  form  was  placed  on  Mayfair,  green 
paper  which  was  of  the  size  shown  above.  The  forms  in 
Appendices  R-?  and  B-3  were  placed  on  8.5  by  11  inch 
whit-e  bond  paper. 
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B-2;  Nurse's  Urgency  Rating  Form  for  Hospital  A 


Who  answered  these  questions? 
j  j patient 

I  | person  accompanying  patient 


Chart  number  _ 
Patient's  Name 


PLEASE  ASK  PATIENT  (OR  PERSON  ACCOMPANYING  PATIENT) : 


How  long  has  the  patient  been  aware  of  the  problem  which 
brought  him/her  to  the  emergency  department  today? 

□  less  than  6  hours 

□  6  to  24  hours 


\  [1  to  7  days 
!  [more  than  a  week 


Does  the  patient  think  that  he/she  may  have  to  stay  in 
hospital? 

(HI no  [J  yes 

i 

IF  NO,  ASK: 

If  the  patient  could  have  obtained  a  doctor's  appointment 
within  the  next  3  days,  would  this  have  been  as  convenient 
for  him/her  as  coming  here  today? 

|  |  no  j  jves 


NURSE'S  URGENCY  RATING 

Using  the  predetermined  classification,  please  indicate 
the  category  to  which,  you  believe,  this  patient  belongs. 

□  emergent  □urgent  □nonurgent 

How  long  have  you  worked  in  an  emergency  department? 

□  less  than  3  months  {□  1  to  2  years 

□  3  months  to  1  year  [Jmore  than  2  years 
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l <- 1 ;  _ N i irsn's  Urgency  Rating  Form  for  Hospital  R 

Who  answered  these  questions?  Chart  number  _ 

|  [ patient 

£j[  person  accompanying  patient 


PLEASE  ASK  PATIENT  (OR  PERSON  ACCOMPANYING  PATIENT) : 

How  long  has  the  patient  been  aware  of  the  problem  which 
brought  him  to  the  emergency  department  today? 

P  | less  than  6  hours  Qj  1  to  7  days 

6  to  24  hours  j~|more  than  a  week 

Did  you  try  to  contact  your  doctor  before  coming  to  the 
hospital? 

□  yes  □  no 

If  yes,  did  he  ask  you 
or  to  the  hospital? 

Qhis  office 


to  come  to  his  office 


j  [hospital 


NURSE'S  URGENCY  RATING 


Using  the  predetermined  classification,  please  indicate  the 
category  to  which  you  believe  this  patient  belongs. 


|  [EMERGENT 


|  [URGENT 


]~1  NONURGENT 


How  long  have  you  worked  in  an  emergency  department? 
j  [less  than  3  months  j  j 1  to  2  years 

|  |3  months  to  1  year  Qmore  t*ian  ^  years 
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B-  A  : 


Physician's  Urgency  Rating  Form 


Chart  Number 
PHYSICIAN'S  RATING  FORM 


Using  the  predetermined  classification, 
please  indicate  the  category  to  which, 
vou  believe,  this  patient  belongs. 


Qfmergfnt  [~~j  urgent 


I~~l  nonurgent 


How  long  have  vou  worked  in  an  emergency 
department? 

Qless  than  3  months  □]  1  to  2  years 
I  [3  months  to  1  year  pjmore  than  2  years 


When  used,  this  form  was  placed  on  nreen,  Mayfair 
paper  which  was  of  the  size  shown  above. 
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B-5:  PANEL  PHYSICIAN'S  URGENCY  RATING  FROM  PATIENT  CHARTS 


Using  the  predetermined  classification,  please  check  the  category 
to  which,  you  believe,  each  of  these  patients  belong. 

Chart  No.  EMERGENT  URGENT  NONURGENT  Chart  No.  EMERGENT  URGENT  NONURGENT 


. 


/ 


-  —4  ■ 


B-6  : 


Data  Coded  for  each  Patient 


Variable 

number 

Columns 

Variable  name 

Codes 

1 

1 

has  "true 
urgency"  rating 

1=  yes 
b=  no 

2 

2-5 

subject  number 

hospital  A:  1-431 
hospital  B:  1-401 

3 

6 

hospital 

1=  hospital  A 

2=  hospital  B 

4 

7-8 

address  by  census 
tract 

5 

9-10 

age 

6 

11 

sex 

1=  male 

2=  female 

7 

12 

marital 

status 

hospital  A: 

not  available  (NA) 

hospital  B: 

1=  single 

2=  married 

3=  other 

8 

13-14 

date 

hospital  A:  25-28 
hospital  B:  17-24 

9 

15-18 

time  in 

10 

19-22 

time  out 

11 

23 

x-ray 

1=  no 

2=  yes 

12 

24 

laboratory  tests 

1=  no 

2=  yes 

13 

25 

physician  who  saw 
patient 

hospital  A: 

2=  part  time  emergency 
physician 

b=  full  time  emergency 
physician 
hospital  B:  NA 
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14 

26 

type  of  patient 

hospital  A: 

2=  Workmen ' s  Comp¬ 
ensation  Board 
(WCB)  patient 

hospital  B:  NA 

15 

27 

disposition 

1=  discharged 

2=  admitted 

28-30 

blank 

16 

31 

who  answered 
questions 

1=  patient 

2=  person  with 
patient 

17 

32 

how  long  with 
problem 

1=  0-6  hours 

2=  6-24  hours 

3=  1-7  days 

4=  more  than  1  week 

18 

33 

hospital  A:  patient 
thinks  he  would  be 
hospitalized 

1=  no 

2=  yes 

hospital  B:  did 
patient  try  to 
contact  doctor 

1=  yes 

2=  no 

19 

34 

hospital  A:  patient 
think  he  could  wait 
to  see  a  doctor 

1=  no 

2=  yes 

hospital  B:  where 
patient  is  to  meet 
doctor 

1=  office 

2=  hospital 

3=  unable  to  reach 
doctor 

20 

35 

clerk's  rating 

hospital  A: 

1=  emergent 

2=  urgent 

3=  nonurgent 

hospital  B:  NA 

21 

36 

clerk's  experience 

hospital  A: 

1=  less  than  6 
months 

2=  6  months  to 

1  year 

3=  1  to  2  years 

4=  more  than  2 
years 

hospital  B:  NA 
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22  37 


23  38 


24  39 


25  40 


26  41 


27  42 


28  43 


29  44 


nurse's  rating 


nurse's  experience 


emergency  physician' 
rating 

emergency  physician' 
experience 


hospital  A:  panel 
physician  I's 
rating 

hospital  B:  panel 
physician  IV 's 
rating 

hospital  A:  panel 
physician  II 's 
rating 

hospital  B:  panel 
physician  V's 
rating 

hospital  A:  panel 
physician  Ill's 
rating 

hospital  B:  panel 
physician  Vi's 
rating 

insufficient 

patient  record 

information 


1=  emergent 

2=  urgent 

3=  nonurgent 

1=  less  than  6 
months 

2=  6  months  to 
1  year 

3=  1  to  2  years 

4=  more  than  2 
years 

1=  emergent 

2=  urgent 

3=  nonurgent 

1=  less  than  6 
months 

2=  6  months  to 
1  year 

3=  1  to  2  years 

4=  more  than  2 
years 


1=  emergent 
2=  urgent 
3=  nonurgent 


1=  emergent 
2=  urgent 
3=  nonurgent 


1=  emergent 
2=  urgent 
3=  nonurgent 


hospital  A: 

0=  not  reported 
1=  reported  once 
2=  reported  twice 
3=  reported  three  times 
hospital  B:  NA 
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